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ABSTRACT 

In 1988, the United States Congress passed 
legislation that created the School Dropout Demonstration Assistance 
Program (SDDAP) . The program provides funding for local educational 
agencies, community-based organizations, and educational partnerships 
to establish effective programs to identify potential student 
dropouts and prevent them from dropping out and to encourage youth 
who have already dropped out to re-enter school. This report, the. 
first of two volumes, presents findings of a two-part evaluation to 
assess the effectiveness of dropout-prevention strategies used by the 
89 projects that were awarded grants. Data were collected through 
annual surveys of all SDDAP projects and indepth studies of 
activities and achievements at 15 selected project sites. The report 
identifies the organizational characteristics of effective dropout 
programs and the program strategies that were most effective in 
preventing students from dropping out of school and in encouraging 
dropouts to reenter school. Outcomes for elementary, middle, and high 
school students are described. The following recommendations are 
offered: (1) use individualized strategies in regular classrooms 
instead of pull-out programs; (2) deliver the special services in 
unobtrusive ways; (3) deliver the special services within a 
supportive climate that includes adults as student advocates; (A) 
provide students with substantive incentives to participate; and (5) 
carefully select, train, and support service-delivery staff. (LMl) 
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Evaluation of Projects Funded by the School 
Dropout Demonstration Assistance Program 

Final Evaluation Repc 
Executive Summary 

In 1988, the U.S. Congress, recognizing the seriousness of the dropout problem in this 
country and the lack of rigorous information about effective dropout prevention programs, 
authorized demonstration programs under Title VI, Part A, of the Augustus F. Hawkins- 
Robert T. Stafford Elementary and Secondary School Improvement Amendments of 1988 
(20 use 3241 £t s£3.), to encourage students to stay in school. The legislation created the 
School Dropout Demonstration Assistance Program (SDDAP), which provides funding for 
local educational agencies (LEAs), community-based organizations (CBOs), and educational 
partnerships to establish and demonstrate the following: 

♦ effective programs to identify potential student dropouts and prevent them from 
dropping out 

♦ effective programs to identify and encourage youths who have already dropped out to 
re-enter school and complete their elementary and secondary education 

♦ effective programs for early intervention designed to identify students at risk in 
elementary and early secondary school 

♦ model systems for collecting and reporting information to local school officials on the 
number, ages, and grade levels of youths not completing their elementary and 
secondary education and reasons why they have dropped out of school 

Eighty-nine projects across the U.S. were awarded two-year grants (which were 
extended to a third year) under the School Dropout Demonstration Assistance Program, 
beginning in fall, 1988. The 89 grants awarded were intended to support demonstrations to 
reduce the number of children who do not complete their elementary and secondary 
education. The projects began operation in September 1988 and were completed in August 
1991. Grants for the first year covered up to 90% of the total program costs (with local 
districts funding the remainder); grants for the second and third years covered up to 75% of 
the total program costs. During the 1988-89 school year, a total of $23,937,359 was awarded 
to the 89 projects; during the 1989-90 school year, a total of $20,269,876; during the 1990- 
91 school year, a total of $19,919,393 was awarded. The total for the three years was 
$64,126,628, with the average grant per yeai* per project amounting to $240,175, which on 
average represented from about 70% to 78% of the total funds available to a specific project 
in any one year. 

To assess the effectiveness of the dropout prevention strategies that these projects used, 
the Department of Education funded a two-part evaluation. The two parts of the evaluation 



were annual surveys of all SDDAP grantees and in-depth studies of activities and 
achievements at selected sites, as described below: 

1. Annual descriptive surveys of SDDAP projects, covering program funding, planning 
and implementation, program organization, types of strategies and approaches used, 
characteristics of participants, and outcome measures. 

2. In-depth studies of selected projects, including implementation site visits and 
extensive data collections via student surveys and school records to compare the 
education-related outcomes of program participants and comparison students. 

The major policy questions addressed by both parts of the evaluation were the following: 

• What are the organizational characteristics of effective dropout prevention programs? 

• What program strategies are most effective in preventing students from 
dropping out of school? In encouraging dropouts to reenter school? 

Overview of the 89 Dropout Demonstration Projects 

Although all 89 projects shared the common purpose of helping students at risk stay in 
school, they we'-e characterized by considerable diversity. Many of the projects, which were 
spread across 31 states and the District of Columbia, were located in major metropolitan 
areas, others operated in smaller towns and rural locations, and a few served specialized 
populations at sites spread widely across the country. The projects also varied in size and in 
the way they were administered. Most operated through their local school districts (76 of the 
89 projects) and offered services within school or altemative-school settings, while others 
operated independently of the school systems and maintained separate facilities (i.e., five 
projects were university-based efforts, five were operated by community-based organ- 
izations, and three were operated by private educational establishments or regional 
associations). 

Improving academic performance and increasing attendance were the most frequent 
project goals, but many projects also aimed to increase self esteem and positive attitudes 
towards school, increase parental involvement and family coping skills, and decrease the 
incidence of discipline problems and school suspensions. The majority of projects offered 
academic skills instruction and counseling, and many offered a wide range of additional 
services, combined in a variety of ways within and across "project components" (i.e., discrete 
programmatic activities for specific groups of students. Many projects were considered to 
offer comprehensive services (i.e., they offered at least one service in each of the following 
service categories: academic, counseling, social support, and, for components serving grades 
6 and higher, vocational/career). Specifically, 41% of projects (i.e., 20 of 49 projects) serving 
K-5 students offered comprehensive services, as did 44% of projects serving grades 6-8 (i.e., 
28 of 64 projects) and 61% of projects serving students in grades 9-12 (i.e., 37 of 61 
projects). 
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Twenty-one of the projects served only students in grades K through 8, 14 projects 
served only students in grades 9 through 12, and the remainder were multilevel projects, 
serving both elementary and secondary populations. Some specifically targeted re-entry 
students, and in all, 43 projects reported including services to re-entry students in addition to 
the services provided to students other levels. One site worked exclusively with the parents 
of at-risk youth. While blacks and whites were the two racial/ethnic groups most served by 
the projects, three projects served Native American populations exclusively, and another 
provided assistance to Hispanic migrant youth. 

In-Depth Studies of Selected Dropout Demonstration Projects 

For the in-depth portion of the SDDAP evaluation, 15 projects were studied in the 1989-90 
school year, and one additional project was studied in the 1990-91 school year. These 
demonstration sites were judged to hold special promise for successful implementation, based on 
reviews of their applications, responses to the 1988-89 survey of grantees, and telephone 
communications with the sites. The sites also represented a diverse cross-section of all SDDAP 
grantees; they were distributed across the United States, from areas as small as Ethete, on the 
Wind River Indian Reservation in Wyoming, to the large urban centers of Los Angeles and New 
York. (In New York, the demonstration site selected for the in-depth study was a community 
school district in the Bronx.) Finally, the sites included in the in-depth portion of the evaluation 
also aimed to serve student populations at varying levels: 

• four sites served elementary school students exclusively — Gushing, Oklahoma; 
Shreveport, Louisiana; Hannibal, Missouri; and Los Angeles, California 

• five sites served middle school students exclusively — Portland, Oregon; Broward 
County, Florida; Denver (Lake), Colorado; San Antonio, Texas; and Bronx, New 



• five sites served high school students exclusively — Des Moines, Iowa; Memphis, 
Tennessee; Aiken, South Carolina; Ethete, Wyoming; and Denver (Discovery), 
Colorado 

• two sites served both high school and re-entry students — Coleman, Texas, and 
Carbondale, Illinois 

Examination of Project Implementation 

Three-day site visits were made to each of the originally selected 15 projects in the spring of 
1990. Information was gathered on the following topics: 

• context for project operations 

• nature, extent, and coordination of services provided 

• fiscal and human resources supporting the project 

• training and experience of project staff 
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• quality of the implementation of each project component (e.g., basic skills instruction, 
support services) 

These data were gathered through interviews with project personnel, district or school 
administrators, project directors, project teachers, ancillary staff, and business partners (if any). 
These data were also gathered through observations of project activities and through focus 
groups that were conducted with project students in grades 5 and above. 

In winter/spring 1991, site visits were made to the four sites at which new student samples 
had been selected the previous fall (including the one new site added to the ev;iluation). During 
these visits, data collected from observations and interviews with project staff and focus-group 
sessions with students were used to produce project descriptions and to rate the quality of project 
implementation at each of the sites on 23 dimensions, covering environmental, institutional, 
psychological, school identification, and school participation domains related to school dropout. 

Finally, in fall 1992, following the period of SDDAP funding evaluated under this contract, 
site visits were made to six projects to study issues related to sustaining and replicating dropout 
prevention approaches. 

Examination of Project Outcomes 

In fall 1989, random samples of approximately 1(X) students were drawn from among 
the participants in each of the 15 SDDAP demonstrations selected for the in-depth study. At 
these sites as well, samples of similar students who were not participating in the projects (and 
who, usually, were not eligible for services from the projects because they were in different 
schools) were drawn as comparisons. In 1990-91, additional student samples were drawn at 
three of these original sites and at one new site, all of which showed considerable promise of 
effective performance. Key to the causal inferences about outcomes was the identification of 
matched comparison samples at the sites. 

Outcome measures. This study gathered data on student status — that is, graduation, 
dropout, or continuing enrollment rates — on an annual basis and used these data to compare end- 
of-the-year dropout rates and the average time to dropout among students in the treatment and 
comparison groups. In addition, data were gathered from students and teachers describing 
behaviors that are generally precursors to dropping out — poor academic performance, socially 
disruptive behavior at school, and poor attendance, all of which have been linked to school 
dropout decisions. While the original intent had been to follow all the students selected in 1989- 
90, the timing of the study and changes in the study design, which included the limitation of 
follow-up data collection activities at some sites and the addition of new samples at other sites, 
resulted in somewhat different approaches at different sites. 

Data extracted from school records included the following: 

• end-of-the-year status information — used to compute dropout rates based on school- 
classified dropouts only and dropout rates based on school-classified dropouts as well 



as students who were expelled or who moved without confirmation of re-enrollment 
(middle and high school students only) 

• percent days absent from school (i.e., total excused and unexcused absences divided 
by total days enrolled and re-scaled to more accurately approximate unexcused 
absences only) 

• average yearly grades 

• number of suspensions (i.e., total in-school and out-of-school suspensions) [middle 
and high school students only] 

The data provided by student surveys included the following items: 

• indicators of students' self-esteem 

• students' attitudes toward school and future education plans 

• reports of parental involvement in class/school activities 

• evaluations of school climate 

• perceived performance on school-related activities/tasks 

Information from teacher surveys used in the cases of projects serving children in 
Kindergarten and grade 1 covered the following areas: 

• classroom citizenship 

• classroom participation 

• language development 

• math/pre-math development 

• peer relationships 

• parent cooperation 

• parent involvement 

At the one site that served re-entry students exclusively, students who left the regular 
school program or the re-entry demonstration since 1989-90 were surveyed with specially 
designed unstructured instruments to learn about students' attitudes, personal experiences, 
and plans. A version of this survey was also used in telephone interviews with students who 
left any of the middle or high school SDDAP demonstrations. 

These follow-up student surveys inquired into the following: 

• their reasons for leaving 

• their perception? of improvements that might have caused them to remain enrolled 

• who they spoke to about leaving 

• their reasons for having enrolled in the re-entry program (re-entry site only) 

• their education- and work-related plans 



Finally, because three of the four projects at which new student samples were selected in 
1990-91 required students to apply for admission, brief survey forms were designed that aimed 
at clarifying the various motivational factors (e.g., peer pressure, parental support) affecting 
students* decisions to participate in special school programs. The different motivational survey 
forms included rating scales for the following dimensions: 

• parental support 

• peer support 

• ease/difficulty of finding time to participate 

• ease/difficulty of finding transportation to participate 

• perceived usefulness (generally and for the individual) of participation for gaining 
employment 

• perceived usefulness (generally and for the individual) of participation for enhancing 
school performance and/or learning important skills 

• perceived usefulness of participation for helping one to feel more a part of the school 

Overview of analytical approach. Gain-score analyses (or the analysis of post-treatment 
minus pre-treatment performance) were used to compare the treatment and control samples at all 
the sites. These analyses always controlled for race/ethnicity, gender, and age with respect to 
grade level. Comparisons of dropout rates between the student samples at the various sites were 
made using Fisher's Exact Test, and survival analysis (using Cox Proportional Hazard 
methodology) was used to assess differences between these groups in the amounts of time 
students remained enrolled in school. Like the gain-score analyses, these analyses of dropout 
rates and pattems always control for race/ethnicity, gender, and age with respect to grade level. 

The analysis of data from this evaluation has required that attention be paid to several 
needed adjustments. For example, controlling for the effects of selected demographic variables in 
assessing differences between the student samples is an adjustment made to minimize the effects 
of extraneous differences among the groups on outcome variables of interest. Similarly, the use 
of gain-score analysis is intended to minimize the effects of pre-treatment differences between 
the samples on these outcome measures. In addition, adjustments are made for less than complete 
response rates at the sites, unequal demographic distributions between student samples at the 
sites, skewed distributions of selected variable values, and imprecise estimates of numbers of 
unexcused absences. 

General Observations Related to Dropout Prevention 

It is important to emphasize that the findings from this evaluation arc based on an 
aggregation of analyses of many different aspects of widely varying dropout prevention projects. 
Specifically, we have attempted to generalize themes from individual results at particular sites; it 
is not the case that all of these themes are based on similar findings replicated across all the 



demonstrations. In addition, our assessment of project success has been based, in the main, on 
the relative assessment of student performance; that is, our judgments typically are made relative 
to a comparison group and may not at all reflect absolute improvements in the performance of 
individual students. For these reasons, there is a fragility in the results that must be emphasized. 
Keeping this in mind, it is possible to make several observations based on the common 
experiences and achievements of the 16 SDDAP demonstration projects that participated in the 
in-depth portion of the study. The first three of these observations relate to the organizational 
characteristics of dropout prevention programs, and the five following observations relate to 
effective dropout prevention strategies. 

Organizational Characteristics 

• The more complex the organizational structure of a dropout prevention initiative 
(i.e., the greater tendency toward restructuring or non-school-based coordination of 
services), the longer the time period that is likely to be required for start-up and the 
less likely it is there will be evidence of gains for students in the short-term. 

• Coordination of services has the potential to increase the services that are available, 
but such efforts require joint planning and review sessions to be successful, and they 
may require increased funding to maintain project efforts. 

• Providing an array of complementary services (i.e., comprehensive services) may be 
the most effective way of meeting the needs of students at risk of school failure. 

Effective Dropout Prevention Strategies 

• Counseling services and adult advocacy for students are key elements of any 
particular dropout prevention initiative. 

• At the elementary level, providing after-school tutoring and enrichment that are 
directly related to in-ciass assignments and having in-class adult friends (e.g., trained 
volunteers or helpers) appear to be effective approaches. 

• At the middle level, team teaching strategies, flexible scheduling, heterogeneous 
grouping of students, and provision of as-needed counseling assistance are especially 
useful strategies. 

• At the secondary level, paid-work, embedded in activities that prepare and monitor 
students' on-the-job experiences, appears to be a critical component to keeping 
students in school. 

• In programs where dropout recovery is an emphasis, flexible class schedules assist 
students who need to work or meet personal commitments during regular school 
hours. 



Recommendations for Implementing 
Dropout Prevention Programs 

Based on our observations and consideration of previous research findings, we have 
formulated yive recommendations related to the implementation of dropout prevention 
projects, which are presented and described below. It is imponant to stress once again that 
these recommendations are the result of generalization from individual results at particular 
sites, and so must be regarded with appropriate caution. 

Recommendation 1: Put the services in rather than pull the students out 

In the 1990-91 survey of SDDAP grantee effons, 50% of K-8 projects, 14% of grade 
9-12 projects, and 60% of multilevel projects (i.e., projects serving students at various grade 
levels) were found to be using pull-out instructional strategies. (These strategies typically include 
removing students from their regularly scheduled classes to permit their participation in special 
instructional programs.) Overall, from 50% to 75% of the projects using pull-out strategies at 
each of these grade levels reported them to be of high importance. 

The most likely rationale for taking this instructional approach is that it is much less 
efficient to build into regular classrooms the types of special activities, often individualized or 
one-to-one activities, that (only) specific students may require. In addition to possibly allowing 
several students at once to receive the special instruction-related service from a single teacher, 
removing students with special needs to another classroom for a period, for example, would free 
the teacher in the regular classroom to continue with scheduled assignments. When large 
numbers of the students being taught in classrooms are in need of special instructional services, 
however, pulling out only selected students for extra help is likely to be much less effective than 
reorganizing the general classroom instructional approach. 

In this in-depth evaluation, the dropout demonstration projects that utilized pull-out 
strategies achieved less convincing patterns of desired student outcomes than did other projects. 
In each case, one or two students were pulled from their regularly scheduled classes for all or 
part of an instructional period. In two of the three cases there was the issue of stigma associated 
with the pull-out process, and in the one case where students seemed to look forward to being 
pulled, at least some of the teachers reportedly resented the tutors who called for students. At one 
site, pulled students did not receive credits for their special classes, and at all the sites utilizing 
this strategy the question was raised concerning what students were missing during the periods 
they were taken from their regular classrooms. 
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Recommendation 2: Deliver the services without calling attention to the fact 

that special services are being provided 

Stigma is likely to be associated with periodic pull-outs from regular classes because 
pulling students often signals to peers that special services are being provided. While it is 
possible, of course, for schools to create such a climate of community among students that even 
pull-outs are regarded as a natural pan of the normal day at school, this is apt to be quite difficult 
to accomplish. At the same time, putting special services into the classroom does not by itself 
entirely solve the problem of stigma. In any number of ways, the most well-intentioned teacher 
can call attention to the fact that he or she is providing tailored instructional assistance to one 
student or to a small group of students in the midst of the larger classroom, which may spur 
teasing or increase the resentments on the parts of peers. What is needed in classrooms in these 
situations, ideally, also is a sense of community among teachers and students, such that every 
student understands and respects the learning-related needs of others and the learning 
opportunities that are provided. Short of this, schools must explore various unobtrusive ways of 
introducing opportunities to learn that are appropriate to the learning-related needs of students. 

Of those demonstrations participating in the in-depth study that recorded gains for students 
relative to the comparison groups, all found ways to subtly introduce special services by either 
(1) entirely avoiding ability grouping as a prerequisite for receiving special services or (2) 
increasing the perceived benefits to students of receiving the special services to such an extent 
that the negative perceptions of grouping were effectively canceled. At some of these sites, the 
roles of adult mentors in the classroom and counselors on campus were defined carefully to 
make it appear as if these individuals were there for all students. One project randomly assigned 
students to the demonstration condition and then altered the regular instructional program for all 
Uiese student-panicipants. When certain participants in this project were provided bilingual 
instruction to build their English skills, for example, other participants received bilingual 
instruction to further develop their verbal skills in Spanish. In those cases where apparently 
effective projects did engage in clustering at-risk students, attempts were made to create 
alternative school environments that would be perceived as providing opportunities for learning 
and work that were simply unavailable in the regular school situation. These opportunities 
included paid-work and the chance to rely on project staff to help with the full range of personal 
problems. 

Recommendation 3: Deliver the services within a supportive climate that 

includes adults as student advocates 

Many researchers have noted the importance of building supportive, caring climates around 
students at risk, and the results from the SDDAP demonstration confirm previous findings. In 
several of our in-depth study sites, students' reliance on their teachers and counselors to help 
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with personal problems evidenced the son of close, caring relationships that appear to be 
necessary for achieving successful results in terms of school performance. In most of the more 
effective projects, developing especially close relationships among staff and students was a 
priority, and, in two cases, these effons resulted in documented student perceptions of the 
improved quality of their school climates. At two other sites, the counselors and outreach 
specialists also served as student advocates, interceding on behalf of students with teachers and 
sometimes with their parents. This theme of care, concern, and advocacy, which also runs 
consistently through the literature on working with students at risk, was a common thread among 
the SDDAP demonstrations that achieved positive student outcomes. 

Recommendation 4: Provide students with substantive incentives to participate 

The SDDAP demonstrations seem to provide rather consistent evidence of the salience of 
student incentives for achieving outcomes related to dropout prevention. No matter whether 
students were required to apply for the dropout prevention programs or simply were selected by 
school officials to receive the services, substantive incentives appeared to be effective. At the 
elementary level, these incentives for students included help in seeing that their assigned 
homework was completed correctly and in timely fashion. At the middle school level, these 
incentives included counselors serving students as adult friends, with whom students could share 
any problems and ask any questions at any time. Finally, at the secondary level, the projects that 
retained students in school provided paid-work and vocational training opportunities.for students 
who remained enrolled or completed their project commitments in good standing. 

These findings are consistent with a solid body of evidence; in fact, incentives have, for the 
most part, been found wanting in evaluation and research studies of students at risk only when 
they have involved deferred rewards. Our emphasis on perceptible, extrinsic rewards for students 
should not be interpreted, however, as a criticism of efforts to socialize youth to respond to 
internal motivations to remain enrolled or to perform well in school. Rather, in the absence of 
such internal motivations, substantive incentives provided by school officials may provide the 
initial encouragement necessary for many students to begin to understand and to internalize 
school-related values. At the same time, incentives must be integrated with the instructional 
program and, in the cases of secondary students in particular, preparation and follow-up 
activities are likely to be needed to ensure the desired results of such strategies as providing paid- 
work experience. 

Recommendation 5: Carefully select, train, and support the staff persons 

providing the services 

In the 1990-91 survey of the SDDAP grantees, 85% at the K-8 level reported having some 
staff with special training; 26% reported these staff had received training prior to working on the 
project, and 67% reported staff had received training while working on the project At the same 
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time, almost 30% of these grantees reported that the shortage of trained staff was their greatest 
obstacle to providing services to students. Successful dropout prevention efforts arc ones that 
select staff carefully and provide initial staff orientation that is more than description of the 
project aims and staff roles. Following orientation, these projects continue to provide skill- 
building opportunities, counseling, problem-solving sessions, and motivational aids to staff to 
maintain necessary focus on key goals and necessary interests and abilities in providing services. 
These types of staff supports appear particularly important when the prevention program 
involves the combination of various services ( e.g., when the program is comprehensive), and 
they seem to become critical when the program represents a new direction from the more 
traditional, regular school program. 

In one of the more successful sites, for example, only teachers with experience in working 
with students at risk were asked to apply for positions. Upon application, these teachers were 
required to visit the alternative school and to talk with the project director and other staff akeady 
selected about the work environment and the project purpose. Only after teacher-candidates were 
debriefed following these visits and expressed themselves still interested in positions were they 
advanced to the actual selection pool. In addition, orientation at this site was really a group 
affair, with teachers sharing their ideas as they welcomed new staff into the family. Further, all 
staff associated with the project were actively encouraged to attend skill-building workshops 
related to methods and curricula for students at risk, and periodic meetings of staff throughout 
the year were specifically arranged to promote the sharing of problems and group brainstorming 
about jxjssible solutions. 

In contrast, at two sites where teacher teams were relied upon to provide instructional 
services and periodic counseling to students, the selection of teachers and their orientation, 
training, and continuing support were aspects of the demonstrations that were less well integrated 
into the project design. At one of these sites in one year, staff did manage to create a distinctive 
climate for students that produced reductions in student absences and suspensions; by the 
following year, however, team members had left the project and the team counselor position had 
been eliminated. Perhaps as a result of these events, student accomplishments for that year and 
any continuation of the benefits identified during the previous year were not evident. At the other 
site, teachers showed signs of having accommodated themselves to a team approach, but several 
team members expressed their hopes for transfers out of the program at the end of the school 
year. The building of bridges for the program with other teachers and the central administration, 
which might have been accomplished in part by the efforts of the teacher teams, seemed stalled, 
and the perceived isolation of these teacher teams may have contributed both to their lack of 
interest in continuing as team members and their lack of effectiveness in connecting to teachers 
in the regular school program. Finally, in one (other) site, where teachers seemed to make 
ineffective use of small class sizes of motivated students, there was no monitoring of teacher 
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efforts and, as a result, no attempt was made to help teachers exploit the instructional advantages 
that were provided. 

Sustaining Dropout Prevention Programs 

Factors related to sustaining innovative approaches were identified in visits to three projects 
that had evidenced patterns of success in working with students at risk sometime during the grant 
period. From these visits, three aims were identified that appear to be important in sustaining 
dropout prevention initiatives, and the projects offered several strategies that relate to achieving 
these aims. 

Aim 1: Reinforcing Staff Commitments to the Program (Team Spirit) 

Staff of the projects developed a camaraderie that helped encourage and sustain their 
commitments to the projects. Whatever the energy levels or conflicting concerns of individual 
staff members on particular days, the sense of being part of a team was likely to have helped in 
focusing attention on the shared aims of the project. The individuals who worked at these three 
sites came from a variety of backgrounds, with varying types and levels of experiences in 
teaching and working with students. They all volunteered to be part of the programs, however, 
and knew from the outset they were participating in special efforts. During the first two years of 
the SDDAP grant period at each of these sites, considerable attention was devoted to building 
team spirit. 

The most common strategy used in building staff teams was to provide the staff with time— 
the time to talk with one another, to plan activities and compare notes. 

A second strategy used in team-building was to instill a sense of joint ownership in the 
projects. Given that the staff persons were involved in special dropout prevention efforts, they 
were enlisted as planners and evaluators of the program. 

A third strategy was to attend to the performance of the team as a team. Specifically, 
project managers or principals monitored how well their teams were functioning, were careful to 
praise and characterize project successes as products of team efforts, and did not hesitate to make 
staffing changes when problems arose. Teachers who didn't fit were reassigned, and new 
teachers or counselors were typically brought into the projects after consultation with resident 
staff. 

Aim 2: Keeping Staff Fresh in Pursuit of Dropout Prevention Goals (Challenge) 

Staff at these three sites clearly were challenged to make a difference for students at risk. At 
one site, two programs that had independently assisted students generally for several years were 
called upon to integrate their operations. Similarly, another of the sites was created as the last 
resort for the most at-risk or endangered youth of seven other school districts. Finally, the third 
project set out to remake the middle school experience at a time when 6th-graders, for the first 



time, were to be assigned to middle school campuses. This sense of challenge also characterized 
project efforts to keep staff motivated in their work with students. 

One strategy used to challenge staff was to provide release-time (or travel costs) for 
training and staff development sessions. 

A second strategy was to publicize program achievements, using these achievements to 
build community support and expectations for the projects. 

A third strategy was to introduce new staff to the projects by rotating existing staff into 
other positions. 

Aim 3: Establishing Connections to Existing Programs (Bridging) 

Whether the aim was to marry two long-running programs (with long-running staffs), to 
serve the needs of seven other school districts whose superintendents comprised the board of the 
demonstration project, or to restructure the school experience for one segment of the school 
population, solid connections between extant and new programs had to be made. The 
experiences of these three projects suggest three strategies that may be important in bridging new 
education-related activities with current programs. 

First, the aims of a new program should be related to ongoing activities, and the 
connections should be communicated to all parties. 

Second, other staff should become familiar with the workings of the new program so that 
they may appreciate its challenges and be able to share in program-related discussions. 

Finally, new educational projects should be actively integrated with ongoing school 
programs. Specifically, the danger lies in permitting new programs to become isolatexi. 

Replicating Dropout Prevention Programs 

Three SDDAP sites attempting three distinctive types of replications were studied: 
adaptation of a program developed locally in new sites within a district, implementation of a 
nationally known school restructuring process (i.e., accelerated schools), and adaptation of a 
nationally recognized model program (i.e., the Diversified Educational Experiences ProgramV 
Each of these replication sites was funded at the level requested, and at each of the sites upon 
notification of award, training sessions were conducted in the models to be implemented. 
Following these initial training sessions, each of the replication sites was provided with ongoing 
assistance. Visits to these projects in 1992-93 confirmed their continuing to implement the 
models selected in 1988-89, although, based on the (sometimes rather limited) data collected for 
this evaluation through 1991-92, none of these projects appeared to have achieved a consistent 
pattern of program-related successes for students. 

Three factors importantly affected the replication activities at various of the sites and may 
have inhibited the effectiveness of the projects for students. These factors included: 

• the fit of the models to the replication sites 
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• the extent of principal buy-in to the replications 

• turf considerations 

Model Fit 

At these three demonstration sites, the models chosen for replication were selected by 
persons other than those charged with implementation. In addition, the specific sites at which the 
implementations would occur were in all cases identified initially by persons other than site- 
based representatives (e.g., principals or teachers). At each of the sites, the lack of staff 
involvement in selection and planning for the intervention affected the levels of outcomes 
achieved. In one case, the model that was selected for replication was not appropriate for the 
student population to be served. In another case, the model that was selected demanded greater 
autonomy on the parts of program staff than the district was willing to give. At the third site, the 
schools that inherited the model program did so without any indication, evidence, or 
demonstrated need for the sorts of local investments the model would require. 

Principaii Buy-In 

At the sites where the principals exerted strong program leadership, the programs 
experienced some successes; where principals were only willing to house the programs, as they 
likely were willing to house any program that brought its own funding and would add to the 
available repenoire of services at the site, considerably diminished levels of student 
achievements were recorded. In the case of one of the sites, where the principal was willing to 
get involved in the program after the fact, he learned a good deal about what might be 
accomplished when staff are motivated and when principals suppon this motivation by actively 
seeking funding to continue programs. 

Turf 

"Turf' issues typically arise in the implementation of collaborative programs (e.g., 
interagency coordination efforts). They may also arise in cases where support or assistance is 
required across jurisdictional lines of authority. At two sites, these issues had much to do with 
limiting what the projects were able to accomplish. At one of these sites, the issues of turf were 
subtle. When the project was initially funded in 1988-89, staff at the original model site were 
pleased and excited about their program being tried in different schools. They were also careful 
about becoming personally involved in the replication activities for fear they would be seen as 
intruding or invading the turf of other school staff. As a result, their formal involvement in the 
replication was rather limited. At the other one of these two replications, the problems associated 
with turf were less subtle. Principals at the regular high schools in the county never bought in to 
the new program or to the concept of an alternative site organized around the principles of the 
program. Most of all, they were not about to be forced into altering their own schools' 
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approaches to handling the problems of students at risk; they effectively protected their turfs and 
provided reinforcement to one another. 

A Sampler of Replication Strategies 

The experiences of these projects in replicating model programs suggest several strategies 
that may be useful for future replication efforts. 

First, there is the need to bring together district-level and school personnel at the outset of 
planning for replication efforts. The choice of the program to be replicated should, ideally, be 
discussed and agreed upon by all paities to be involved in the effort, and the choice of the 
replication site should probably be a product of these discussions. 

A second strategy is to encourage regular and ongoing interactions involving district-level 
and school personnel throughout the life of the program— perhaps in the form of a general 
steering committee. 

Third, special efforts must be made to make the program especially meaningful to the 
principals at the replication sites. For example, these principals should be encouraged to attend 
training sessions related to the model program, and, whenever possible, they should be 
encouraged to think of the ways they can use or expand upon the replication experience to gain 
support for their schools generally. 

A fourth strategy for enhancing a replication effort, when more than a single school is to be 
involved in that effort, is to decide upon a mechanism for coordinating activities across the sites. 
For example, a district coordinator for these sites can take the lead in organizing cross-site 
meetings, training sessions, and more informal discussions without fear of turf problems; for 
example, he or she can always say "It's just part of my job...." 

Finally, a fifth strategy is to promote continual improvement in the replication effort. 
Specifically, persons with experience in the model to be replicated should be identified as special 
"resources," and ways should be found to encourage their visiting the replication sites on a 
regular basis over the life of the effort. 

Clearly there is overlap among these five strategies and their intended aims. These types of 
strategies are especially critical, however, in efforts to move beyond the copying of model 
programs to their further development. Indeed, one important goal of replicating any model 
dropout prevention program must be to enhance the robustness of that model through adaptation 
and extension of its features. 
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Chapter 1. Introduction 



In 1991, 85% of 19- to 20-year-olds had completed high school (National Governor's 
Association, 1992). While this percentage is an improvement over the 1975 figure of 83%, it 
nonetheless means that a substantial number/percentage of our young people dropped out 
before completing high school. Dropping out represents not only lost opportunities for 
education and personal enrichment for the individual young men and women who leave 
school early, it also means a loss for the society as a whole in reducing the pool of skilled 
workers, increasing welfare and unemployment costs, and creating a less informed citizenry. 
In dollar amounts, it is estimated that dropping out of school costs about $100,000 in lost 
lifetime earnings for each dropout. With about 500,000 students leaving secondary school 
each year, this translates to $50 billion in lost earnings alone (Barro & Kolstad, 1987), 
without factoring in social costs. 

Information is lacking about the precise causes of student dropout, but much is known 
about the various chaiacteristics of students, schools, and community environments that may 
serve as precursors of dropping out. For example, Barro and Kolstad found that racial or 
ethnic group and socioeconomic status are factors associated with premature exit from 
school, with members of minority groups from poor, non-English-speaking homes more 
likely to drop out. Table 1 , for example, shows the relation of dropping out to race or ethnic 
group and gender for two age groups: 16-17 and 18-19 year-olds. As the table indicates, the 
dropout rates for males and females appear to vary as a function of ethnicity and age. 
Among 16-17 year-olds, for example, white and Hispanic females had dropped out at slightly 
higher rates than white and Hispanic males. For Black youtli in this age range, however, 
males had dropped out at higher rates than females. Among 18- to 19-year-olds, white 
females also had dropped out at slightly higher rates than males, but Hispanic males had 
dropped out at much higher rates than Hispanic females (i.e., 38.7% versus 26.7%). For 
black youth in this age range, females had dropped out at higher rates than males, reversing 
the pattern found for 16-17 year-olds. In general, black and Hispanic youth dropped out at 
higher rates than white youth within the same age ranges. The dropout rate among Hispanic 
young people is shown to be particularly high, more than doubling the rates for the other 
groups within the same age ranges and gender categories save one— in comparison to black 
males aged 16-17 years, the dropout rate for Hispanic males in this age group is slightly less 
than twice as high (i.e., 15.1% versus 8.1%). 

Much of the difference in rates by ethnic groups, however, can be accounted for by 
disproportionate poverty rates. Because high school dropouts are more likely to be members 
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Table 1 

High School Dropouts as a Percentage of the Population, 
by Age, Race or Ethnic Group, and Gender* 







All Races 


White 


Black 


Hisoanic H 


Age 
range 


Total 


Male 


Female 


Total 


IVlale 


Female 


Total 


IVlale 


Female 


Total 


Male 


Female 


16-17 


6.0% 


5.6% 


6.3% 


4.6% 


4.1% 


5.2% 


7.3% 


8.1% 


6.5% 


15.8% 


15.1% 


16.5% 


18-19 


13.3% 


13.7% 


13.0% 


10.1% 


9.9% 


10.3% 


14.0% 


12.3% 


15.6% 


32.7% 


38.7% 


26.7% 



♦ Dropouts are persons who are not enrolled in school and who are not high school graduates. Source: U.S. 
Department of Education, National Center for Educational Statistics, U.S. Digest of Educational Statistics. 
1992. Data for 18- to 19-ycar-olds are for October 1989 and based on data from the U.S. Department of 
Commerce, Bureau of the Census, Current Population Reports, Scries P-20, Nos. 222 and 429; and unpublished 
data. 



of racial or ethnic minorities who are from poor, non-English-speaking homes, the dropout 
problem is becoming increasingly urgent, as the U.S. population changes. Estimates show 
that the minority school-aged population is increasing disproportionately. While the overall 
U.S. population of young people under 18 is expected to increase by about 4% between 1988 
and 2020, the percentage of these students who are white is expected to decline by about 
27%, the number of black students is expected to increase by 9%, and the number of 
Hispanic students is expected to increase by 300% (Natriello, McDill, & Pallas, 1990). 

The decision to drop out of school is itself preceded by a number of conditions or 
circumstances. Ekstrom and colleagues found that, in addition to lower SES and minority 
race or ethnicity, students who eventually dropped out exhibited the following 
characteristies, in comparison with their peers: 

• lower test scores 

• lower grades in school 

• less frequent completion of homework 

• more disciplinary problems in school 

• homes with weaker educational support systems, e.g., fewer study aids, single-parent 
families, mothers with lower levels of formal education 

In addition, dropouts differed in out-of-school behavior; for example, they were 
somewhat less likely to discuss their education-related experiences with their parents, and 
they tended to have lower educational expectations and lower self-concepts. Finally, students 
who left school prematurely reported a number of reasons for their actions: did not like 
school (33%), poor grades (33%), offered job and chose to work (19%), getting married 
(18%), could not get along with teachers (157o), had to help support family (1 1%), pregnancy 
(11%), and expelled or suspended (10%) (sec Ekstrom, Goertz, Pollack, & Rock, 1987). 
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Much is known about which students are likely to drop out. In fact, Natriello and 
colleagues observed that "most scientific research in the area [of high school dropouts] has 
focused on the student precursors of dropping out rather than on the programs that may 
reduce these risks..." (Natriello, Pallas, McDill, McPartland, & Royster, 1988). It has been 
suggested, for example, that students who more closely identify with the norms and values 
of the schools they attend are more likely to develop lasting commitments to graduation- 
related goals (see Tinto, 1975; Finn, 1989); however, just how schools are to succeed in 
increasing the identification process has rarely been spelled out with certainty. 

To increase understanding in this area, education practitioners and researchers have 
been examining the experiences of specific dropout prevention efforts to offer general 
recommendations for program strategies. For example, in 1987, superintendents from 32 
major urban public school districts set forth six strategies for ameliorating dropout problems: 
intervene early, create a positive school climate, set high expectations, select and develop 
strong teachers, provide a broad range of instructional programs, and initiate collaborative 
efforts involving families and community agencies (OERI Urban Superintendents Network, 
1987). Similarly, in the mid-to-late 1980s, researchers began to synthesize apparently 
effective program elements in several key areas of focus for educating youth at risk of 
failure, e.g., academic and remedial strategies, integration of academic instruction into 
vocational education, and provision of a wide range of counseling and social services (e.g., 
GAO, 1987; Orr, 1987). Along with this increased study of instructional approaches, 
researchers began providing insights into the aspects of school structure and climate that can 
affect students' willingness to identify with and participate in school activities. These 
insights have led to calls for flexible scheduling, cooperative as opposed to competitive 
learning approaches, and positive teacher and administrative attitudes toward shared school 
goals (see, for example, Wehlage & Rutter, 1986; Lipsitz, 1984). While a number of 
strategies and approaches seem to have the potential to reduce dropout rates, rigorous 
evaluation data on the success — or failure— of these strategies have not been available. The 
link between intent and actual results produced has not been well documented. In sum, by 
1988, enough had been learned to design and rigorously test specific combinations of dropout 
prevention approaches in the field. 

Funding of Projects and the Call for a National Evaluation 

In 1988, the U.S. Congress, recognizing the seriousness of the dropout problem in this 
country and the lack of rigorous information about effective dropout prevention programs, 
authorized demonstration programs under Title VI, Part A, Section 6004 of the Augustus F. 
Hawkins-Robert T. Stafford Elementary and Secondary School Improvements Act (20 USC 
3241 £l5£a-), to encourage students to stay in school. The legislation created the School 
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Dropout Demonstration Assistance Program (SDDAP) for Fiscal Year 1988, which provided 
funding for local education agencies (LEAs), community-based organizations (CBOs), and 
educational partnerships to establish and demonstrate the following: 

• effective programs to identify potential student dropouts and prevent them from 
dropping out 

• effective programs to identify and encourage children and youth who have already 
dropped out to re-enter school and complete their elementary and secondary 
education 

• effective programs for early intervention designed to identify students at risk in 
elementaiy and early secondary school 

• m.odel systems for collecting and reponing information to local school officials on the 
number, ages, and grade levels of children and youth not completing their elementary 
and secondary education and reasons why they have dropped out of school 

Eighty-nine projects across the U.S. were awarded two-year grants (which were 
extended to a third year) under the School Dropout Demonsu-ation Assistance Program 
beginning in Fall 1988. The 89 grants awarded were intended to support demonstrations to 
reduce the number of children who do not complete their elementary and secondary 
education. The projects began operation in September 1988 and were completed in August 
1991. Grants for the first year covered up to 90% of the total program ccsts (with local 
districts funding the remainder); grants for the second and third years covered up to 75% of 
the total program costs. During the 1988-89 school year, a total of $23,937,359 was awarded 
to the 89 projects; during the 1989-90 school year, a total of $20,269,876; during the 1990- 
91 school year, a total of $19,919,393 was awarded. The total for the three years was 
$64,126,628, with the average grant per year per project amounting to $240,175, which on 
average represented from about 70% to 78% of the total funds available to a specific project 
in any one year. 

To assess the effectiveness of the dropout prevention strategies that these projects used, 
the Depanment of Education funded a two-pan evaluation. The two parts of the evaluation 
included annual surveys of all SDDAP grantees and the in-depth studies of activities and 
achievements at selected sites, as described below. 

1. Annual descriptive surveys of SDDAP projects, covering program funding, 
planning, and implementation, program organization, types of strategies and 
approaches used, characteristics of participants, and outcome measures 

2. In-depth studies of selected projects, including implementation site visits and 
extensive data collections via student surveys and school records to compare the 
education-related outcomes of program participants and comparison students 

T'lc major policy quej tions addressed by both parts of the evaluation were the following: 

• What are the organizational characteristics of effective dropout prevention 
programs? 
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• What program strategies are most effective in preventing students from 
dropping out of school? In encouraging dropouts to reenter school? 

Annual Descriptive Survey 

Three annual surveys of all SDDAP grantees were conducted, one at the close of each 
year of the grant period. Recognizing that each grantee might be serving students at various 
grade-levels within different project components, attempts were made to obtain information 
on the goals, services, and outcomes associated with each component. In some cases, projects 
submitted separate, completed survey forms for each component or "sub-project" in 
operation at a site; these were aggregated to provide a summary description for each grantee. 

The surveys sought specific information on the planning and start-up experiences of the 
projects, as well as the organization of activities and the characteristics of project staff and 
student participants. All projects submitted completed surveys for the 1989-90 and 1990-91 
school years. In 1988-89, the first year of the grant period, the response rate was 87%, or 77 
out of 89 projects responding. Selected results from the 1990-91 annual survey are presented 
below. 

Overview of the 89 Dropout Demonstration Projects 

Although all 89 projects shared the common purpose of helping students at risk stay in 
school, they were characterized by considerable diversity. Many of the projects, which were 
spread across 31 states and the District of Columbia, were located in major metropolitan 
areas, others operated in smaller towns and rural locations, and a few served specialized 
populations at sites spread widely across the country. The projects also varied in size and in 
the way they were administered. Most operated through their local school districts (76 of the 
89 projects) and offered services within school or alternative-school settings, while others 
operated independently of the school systems and maintained separate facilities (i.e., five 
projects were university-based efforts, five were operated by community-based 
organizations, and three were operated by private educational establishments or regional 
associations). 

Improving academic perform.ance and increasing attendance were the most frequent 
project goals, but many projects also aimed to increase self esteem and positive attitudes 
towards school, decrease the number of dropouts, increase parental involvement and family 
coping skills, and decrease the incidence of discipline problems and school suspensions. The 
majority of projects offered academic skills instruction and counseling, and many offered a 
wide variety of additional services, combined in a variety of ways within and across project 
components. Many projects were considered to offer comprehensive services (i.e., they 
offered at least one service in each of the following service categories: academic, counseling, 
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social suppon, and, for components serving grades 6 and higher, vocational/career). 
Specifically, 41% of projects or project components serving K-5 students offered 
comprehensive services, as did 48% of projects or project components serving grades 6-8 and 
61 % of projects or project components serving students in grades 9-12. 

Twenty-one of the projects served only students in grades K through 8, 14 projects 
served only students in grades 9 through 12, and the remainder were multilevel projects, 
serving both elementary and secondary populations. Some specifically targeted re-entry 
students, and 43 projects in all reported providing services to re-entry students that may have 
been in addition to the services provided to other levels. One site worked exclusively with the 
parents of at-risk youth. While blacks and whites were the two racial/ethnic groups most 
served by the projects, three projects served solely Native American populations, and another 
provided assistance to Hispanic migrant youth. 

Project Goals 

Projects were asked to specify their five most important goals. Improving academic 
performance and increasing attendance were the goals most frequently mentioned; many 
projects also aimed to increase self esteem and positive attitudes towards school, decrease the 
number of dropouts, increase parental involvement and family coping skills, and decrease the 
incidence of discipline problems and school suspensions. As might be expected, decreasing 
numbers of dropouts and providing career awareness activities, vocational training, and job 
placement were goals that increased in frequency in the 34 projects serving predominantly 
older students; on the other hand, goals of improving self esteem and fostering more positive 
attitudes towards school were more common among the 46 projects serving primarily 
younger students. The SDDAP goal related to development of model local reporting systems 
was addressed by less than one-fifth of the projects. 

Project Participants 

Recruitment. Among projects serving only elementary and middle school students, 
teacher recommendation was the criterion most frequently used for participant identification 
and selection. This was followed by poor attendance, low test scores, and being over-age for 
grade. For projects serving only secondary school students, the most frequently used criteria 
were poor attendance/truancy and number of courses failed. However, multiple identification 
and selection criteria were the norm for all projects. Over 75% of the projects used five or 
more criteria for each of these purposes, and only a handful restricted themselves to a single 
criterion for either purpose. 

Demographics. During the 1990-91 school year, the 89 SDDAP projects served a total 
of 36,97.5 black students, 21,218 white (not Hispanic) students, 17,447 Hispanic students, 
2,206 students of other racial/ethnic origins (American Indian, Native Alaskan, Asian, and 
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Pacific Islander), and 4,615 additional students whose race was not identified. Slightly more 
males than females were served by the SdDAP projects, and special education students were 
the sole population for two projects. 

Services Offered 

Projects were surveyed as to the types of services offered within grade-level 
components because, in many instances, programs offered to different grade levels were 
quite autonomous and discrete, despite being under the umbrella of a single project. The 
services offered by the projects can be grouped into four categories: academic skills 
instruction, counseling services, social support services, and vocational/career awareness 
services. 

Academic skills instruction. Nearly three quarters of the project components serving 
elementary or middle school students offered instruction in basic skills, while 71% of the 
project components serving specifically re-entry students and 66% of those serving grades 9- 
12 also offered this service. In addition, 62-65% of the project components at each grade 
level provided supplementary instruction in regular courses, while substantial numbers (i.e., 
30% and 48%, respectively) of the components serving secondary school students also 
offered GED instruction and remediation to pass standardized tests for promotion. Among 
methods of instruction, small-group instruction and individualized instruction were used 
most frequently and were most likely to be rated of high importance by the projects overall. 

Counseling and social support services. Personal and/or family counseling services 
were offered by 71% of the project components serving grades K-5 and nearly 90% of the 
project components serving each of the older grade levels. Generally the services were 
offered throughout the school year, but only for one or two hours a week on the average. In 
contrast, fewer than half the project components serving the various grade levels offered 
health services to participants as part of their program services, while other social support 
services were somewhat more common, particularly among project components serving 
grades 9-12 and re-entry students. Approximately 60% of these project components offered 
other social services, and over 40% also offered parenting classes for teens. 

Vocational/career services. Data on vocational/career services were collected only for 
project components serving students at grade 6 or above. At these levels, career 
awareness/job counseling was the most frequently offered service (68% of the 6th-8th grade 
components, 74% of the 9th-12th-grade components, and 69% of the re-entry components). 
Vocational training, work study/job internships, and job placement services were each 
offered by a third or more of the 9th-12th-grade and re-entry components, while a smaller 
number of 6th-8th-grade components offered vocational training and work study. 



Parental Involvement 

Across all projects, providing special communications about their child's school 
progress and involving parents in the school's major disciplinary actions towards their child 
were the methods most frequently used to encourage parental involvement, and these also 
tended to be rated as highly important for program success. 

Types of Outcome Measures and Reported Benefits 

Improving academic achievement. Sixty-two of the 84 projects that reported 
information on major goals specified a goal of improving academic skills or performance. 
Regardless of whether or not they specified this outcome as a goal, the great majority of the 
projects reported colleciing standardized test data in reading and mathematics. The majority 
of projects collecting test data also reported that the data showed benefits for their projects. 

Increasing promotions/course passed. Only 18 projects explicitly listed increasing 
rates of promotion or courses passed as one of their five major goals. Projects with this 
explicit goal were somewhat more likely to report collecting data on promotions and courses 
passed, and somewhat more likely to report seeing benefits on these measures. The 
differences are not great, however, and the data were collected and appeared to show benefits 
for the majority of projects, irrespective of whether this was a goal of the project. 

Increasing attendance. Fifty-five projects listed increased attendance as one of their 
five major goals. Virtually all projects reported collecting data on attendance, whether or not 
increasing attendance was given as one of their five major goals. Furthermore, the majority 
of projects in each grade grouping reported seeing benefits on this measure. It would appear 
that benefits were observed somewhat more frequently among projects listing attendance as a 
goal. However, here as elsewhere, it was not possible to distinguish between projects where 
the data were not (yet) analyzed versus projects where the data were analyzed but failed to 
show the desired results. 

Decreasing discipline problems. Twenty-four projects listed decreasing discipline 
problems as one of their five major goals, but the rates at which projects reported collecting 
data on discipline problems or observing benefits if data were collected do not appear to be 
Strongly influenced by having this as a goal. Only at the lower grade levels is there a 
suggestion of increased benefits among projects with the explicit goal of decreasing 
discipline problems. 

Decreasing dropout rates. Twenty-nine projects reported reducing dropouts as one of 
their five major goals. Collection of dropout data was very common irrespective of whether 
projects listed this as a goal. There is some suggestion that gains were more frequent among 
projects that held this goal, but the differences are only striking at the lowest grade levels, 
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and since dropping out is rare among primary students in any case, it is difficult to estimate 
the magnitude of these reported benefits. 

Involving parents. Finally, 27 projects listed increasing parental involvement and/or 
family coping skills as one of their five major goals. The projects in general reported a high 
frequency of family-oriented activities, particulariy family outreach, but the provision of 
these services did not appear to vary significantly according whether this was a goal. 

Project Continuation 

Finally, the projects were asked to comment on the probable status of project activities 
once the final year of the SDDAP grant was completed. Nineteen percent of the projects 
expected the project to terminate, 25% expected the project (or some of its components) to 
continue at a reduced level, and 27% expected the project to continue at its present level or to 
expand. The remaining 29% reported that they were seeking another federal grant and/or 
looking for monies from other sources. At least 42% of these projects seeking continuation 
funds apparently expected their project to be able to continue, at least at a reduced level, 
regardless of the outcome of their on-going fund-raising efforts. 

Several of the projects that reported they expected to terminate or reduce services 
pointed out that certain processes or products developed by their projects would continue to 
be used outside of the scope of the project. For example, one project reported that participant 
schools would continue to use the project's computerized system to monitor and improve 
attendance, while another project reported that each participant school had developed a 
strong, committed decision-making team that would continue to meet and plan school 
improvement initiatives. 

In-Depth Studies 
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Selection of Study Sites 

For the in-depth portion of the SDDAP evaluation, 15 projects were selected in the 
1989-90 school year and one additional project was selected in the 1990-91 school year. 
These demonstration sites were judged to hold special promise for successful 
implementation, based on reviews of their applications, responses to the 1988-89 survey of 
grantees, and telephone communications with the sites. The selection criteria included the 
reported numbers of hours per week that students were served by one or more project 
components (a minimum of 15 hours per week), the reported numbers of students served (a 
minimum of 75), length of service time (a minimum of 15 weeks or one semester), and the 
diversity of project strategies in use. The selected demonstration sites represented 
geographically diverse areas and served students at every level: kindergarten and elementary 
grades, middle schools, and high schools and re-entry programs. The one site that was added 
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to the evaluation in 1990-91 represented an attempt at extending a promising SDDAP 
middle-school program (which had already been selected for the in-depth evaluation in 
1989-90) to one of the high schools in the district. Because of the selection criteria that were 
employed, all of the in-depth study sites addressed aims related to the first three of the four 
SDDAP funding criteria (see pp. 21-22). None of the projects selected had as a primary goal 
the development of a model reponing system, which was the founh SDDAP activity 
authorized. 

Following the 1989-90 school year, data collection activities were intentionally limited 
at four sites, one of which was eventually dropped from the evaluation in 1990-91. 
Methodological problems and the lack of available student-related information for key 
indicators at these sites made it necessary to limit the approach to data collection that had 
been planned. In 1990-91 as a result, resources were available to select new participants (and 
comparison students) at three of the originally selected demonstration sites that were showing 
considerable promise of positively affecting students' school experience. In addition, 
participant and comparison samples of students were selected from the new high school 
project that was begun in 1990-91 and modeled on the SDDAP middle school project 
already operating successfully in the district. 

Examination of Project Implementation 

Site visits. Three-day site visits were made to each of the originally selected 15 projects 
in Spring 1990. Information was gathered on the following topics: 

♦ context or environment for project operations 

♦ nature, extent, and coordination of services provided 

♦ fiscal and human resources supponing the project 

♦ training and experience of project staff 

♦ quality of the implementation of each project component (e.g., basic skills instruction, 
support services) 

These data were gathered through interviews with project personnel, district or school 
administrators, project directors, project teachers, ancillary staff, and business partners (if 
any). They were also gathered through observations of project activities and focus groups 
that were conducted with project students in grades 5 and above. Detailed project 
descriptions were developed based on this information and were updated at least annually 
with each of the sites by telephone or by personal visit to the site. 

In winter/spring 1991, implementation study site visits were made to the four sites at 
which new student samples had been selected the previous fall (including the one new site 
added to the evaluation). During these visits, data collected from observations and interviews 
with project staff and focus-group sessions with students were used to produce project 
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descriptions and to rate the quality of project implementation at each of the sites on 23 
dimensions, covering environmental, institutional, psychological, identification, and 
participation domains related to school dropout. Finally, in fall 1992, following the period of 
SDDAP funding evaluated under this contract, site visits were made to six projects to study 
issues related to sustaining and replicating dropout prevention approaches. Specifically, three 
projects were visited that had demonstrated patterns of effectiveness during one or more 
years of the evaluation and either were continuing their efforts or had experienced problems 
that resulted in termination of project activities, and three others were visited that had aimed 
to replicate national or local dropout prevention models with varying results . 

Identifying areas of plausible student achievements. Based on the aims, natures, 
intensities, and implementation levels of the service approaches used by these SDDAP 
demonstration projects, areas of "plausible student achievements" were identified for each 
project. These areas typically overiapped with projects' goal statements, but they provided 
more focused estimates of likely impacts against which actual results could be contrasted. 
The reporting of project outcomes in Chapters 3-5 is organized in terms of these areas of 
plausible achievement, although outcomes outside these areas are also presented. Since, in 
some cases, changes in student performance may well be delayed until a particular 
intervention has time to take effect, areas of plausible achievements are identified as either 
short- or longer-term expectations. 

Examination of Project Outcomes 

Selection of students. In Fall 1989, random samples of approximately 100 students 
were drawn from among the participants in each of the 15 SDDAP demonstrations selected 
for the in-depth study. At these sites as well, samples of similar students who were not 
participating in the projects (and who, usually, were not eligible for services from the 
projects because they were in different schools) were drawn as comparisons. At one of the 
sites, school officials had already randomly assigned students to the demonstration and to a 
control group prior to the start of the evaluation; at another site, which focused its efforts 
exclusively on providing re-entry services, a longitudinal design was adopted, and 
approximately 200 students at risk of school dropout were selected as the principal study 
group from among the enrollees in the district's three high schools. These students were then 
followed for three years to determine (1) whether they dropped out, (2) if they did, whether 
they entered the SDDAP demonstration, and (3) if they entered the program, what results 
were achieved. In Fall 1990, additional random samples of new SDDAP project participants 
and comparison students were selected at four of the districts already participating in the in- 
depth evaluation (as described above). 



3$ 



Key to the measurement of outcomes was the identification of matched comparison 
samples at the sites, and a three-step matching procedure was used to select these groups. 
First, tlie manner in which students had been selected for the various treatments was 
determined and used to decide on general rules for locating (or siting) comparison pools. If 
students had been selected at random (as was the case at one site), the control group was 
drawn at random from the same school population (in this case, drawn at random from the 
students already randomly assigned to the control condition). If students were identified for 
service at particular schools by teachers or other school personnel (i.e., without the students' 
knowledge or consent), the control group was drawn from a different school in the same 
district. If students were identified on a district-wide basis for service by school 
professionals, the control group was drawn from a different school district from the same 
geographical area and with as many as possible of the same characteristics (e.g., total 
enrollment). Finally, if students or their parents were required to volunteer for service, the 
control group could be drawn in either of two ways — from among those volunteers whose 
non-acceptance into the program was based solely on the later receipt of their applications, or 
from schools in the same (or a different) school district whose students had not been eligible 
to apply for services. 

The second step in developing matched comparison samples was to ask that school 
personnel examine the possible comparison sites and comparison students allowed under 
these general selection rules and identify pools of control group candidates who they deemed 
to be equally at risk as the students served by the demonstrations— i.e., using the same 
procedures they had used in identifying/selecting students for treatment. This procedure is 
subjective, but no other way to define comparison pools that would ensure greater reliability 
of selection given the constraints was considered practical. 

Finally, the third step involved gathering demographic data for as many o-eatment and 
candidate cono-ol group students as possible. Given the time-frame and the extent of 
availability of various school-based sources of information, four indicators were relied on: 
race/ethnicity, gender, age, and most recent grade level completed. As nearly as possible, 
candidate cono-ol students were matched to the treatment students from within each of the 
subgroups defined by these variables for particular sites. 

At one or two sites, this matching so-ategy was complicated by the fact that information 
on race/ethnicity was considered confidential by school officials. At all the sites, however, 
this strategy served to ensure that sufficient cases were available so that subsequent analyses 
could control for the effects of race/ethnicity, gender, and age/grade differences between the 
groups. Table 2 summarizes the approaches used in identifying the comparison pools at the 
15 sites in 1989 and at the four sites in 1990. 
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Sources 


Numbers of Sites i 




1989 


1990 1 


Same district, different schools 


6 


4 


Same district, same schools 


6 


0 


Different district, different schools 


3 


0 



Tables 3, 4, and 5 present comparative data on the sizes, genders, races/ethnicities, and 
age-grade differentials of the treatment- and control- group samples at the in-depth study 
sites that were selected in fall 1989 (note that treatment and control groups were not selected 
at the one longitudinal study site). Table 6 presents similar data for the sites at which new 
student samples were selected in Fall 1990. 



1 

Gender, Racc/Ethnicity, and Age-Grade Dil 

Selected 


able 3 | 

fcrential for Elementary School Student Samples i 
in Fall 1989 1 


Projects 


Cushing 


Shreveport 


Hannibal 


Los Angeles H 


T 


C 


T 


C 


T 


C 


T 


C H 


Sample Size 


100 


100 


99 


100 


100 


100 


114 


83 


Gender 


















Male 


49 


68 


46 


50 


45 


49 


78 


56 


Female 


51 


32 


48 


49 


55 


51 


36 


27 


No. Missing 






5 


1 










Race/Ethnicity 


















White 


78 


85 


0 


1 


85 


91 


17 


10 1 


Non-White 


22 


15 


93 


99 


15 


6 


85 


66 1 


No. Missing 






6 


1 




3 


12 




Age Differential 


















Average for Grade 


94 


94 


80 


69 


44 


44 


87 


60 


Too Old for Grade 


6 


6 


12 


31 


56 


53 


14 


5 


No. Missing 






7 






3 


13 


18 



33 

29 



Table 4 

Gender, Race/Ethnicity, and Age-Grade Differential for Middle School Student Samples 

Selected in Fall 1989 



Projects 


Portland 


Broward Co. 


Denver-Lake 


San Antonio 


New York 


T 


C 


T 


C 


T 


C 


T 


C 


T 


C 


Sample Size 


97 


99 


88 


71 


107 


113 


100 


100 


110 


86 


Gender 






















Male 


46 


45 


43 


52 


61 


60 


61 


55 


53 


37 


Female 


51 


54 


45 


16 


46 


53 


39 


45 


51 


49 


No. Missing 








3 










6 




Race/Ethnicity 






















White 


71 


74 


18 


31 


20 


22 


23 


37 


. 3 


8 1 


Non-White 


26 


25 


70 


37 


86 


87 


77 


63 


107 


78 1 


No. Missing 








3 


1 


4 










Age Differential 






















Average for Grade 


68 


69 


32 


7 


77 


81 


31 


20 


65 


51 1 


Too Old for Grade 


29 


30 


56 


64 


30 


32 


69 


80 


36 


35 1 


No. Missing 


















9 





Table 5 

Gender, Race/Ethnicity, and Age-Grade Differential for High School Student Samples 

Selected in Fall 1989 



"1 



Projects 


Des Moines 


Memphis 


Coleman 


Aiken 


Ethete 


T 


C 


T 


C 


T 


C 


T 


C 


T 


C 


Sample Size 


111 


98 


97 


78 


102 


100 


95 


97 


100 , 


100 


Gender 




















Male 


69 


59 


40 


32 


59 


50 


69 


62 


57 


48 


Female 


49 


39 


57 


46 


43 


50 


26 


35 


43 


52 


No. Missing 






















Race/Ethnicity 






















White 


79 


59 


0 


23 


57 


61 


76 


65 


2 


1 


i Non-White 


32 


38 


97 


55 


45 


39 


19 


32 


97 


99 


1 No. Missing 


















1 




1 Age Differential 






















1 Average for Grade 


66 


37 


58 


30 


11 


21 


18 


42 


52 


64 


1 Too Old for Grade 


45 


61 


35 


29 


91 


79 


77 


55 


46 


36 


1 No. Missing 






4 


19 










2 
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Table 6 

Gender, Race/Ethnicity, and Age-Grade Differential for Middle and High School Student Samples 

Selected in Fall 1990 



Projects 


Portland 


Broward Co. 


Memphis 


Denver-Discovery 


T 


C 


T 


C 


T 


C 


T 


C 


Sample Size 


100 


103 


91 


114 


100 


98 


99 


114 


Uender 


















Male 


47 


4Q 


44 


70 


39 


37 


51 


60 


1 Female 


J J 




47 


44 


61 


61 


48 


54 


No. Missing 


















Race/Ethnicity 


















White 


71 


64 


25 


36 


0 


0 


7 


1 


Non-White 


29 


38 


66 


78 


100 


98 


92 


103 1 


No. Missing 


















Age Differential 


















Average for Grade 


78 


72 


46 


44' 


68 


35 


59 


77 1 


Too Old for Grade 


21 


25 


43 


44 


32 


49 


31 


35 


H No. Missing 


1 


6 


2 


26 




14 


9 


2 



These summary descriptions of the treatment and control groups at the various sites 
suggest broad similarities and some differences between the samples chosen at particular 
sites. Lxx)kirtg particularly for cases in which the distributions of gender, race/ethnicity, or 
age differential vary considerably for the treatment and control samples at particular sites, 
one sees evidence that, at several sites, matching proved a less than effective strategy for 
establishing like groups of students. In Gushing in 1989-90, for example, many more boys 
were in the control group than participated in the demonstration program. Similarly, in Aiken 
in 1989-90, fewer students in the control group were too old for their grade. The most 
dramatic example of differences between the treatment and control students at a site is 
provided by the samples for Broward County in 1989-90, where the samples differed 
substantially on each of the matching variables. In fact, this was the result of an error ui 
procedures used by loc^l school officials to identify a comparison pool from computerized 
records. The error was corrected in the selection of the 1990-91 student samples at the site; 
however, in subsequent chapters, findings comparing the performance of the 1989-90 
treatment and control samples in Broward County must be interpreted with caution. 

Analyses of baseline differences on two outcome measures (i.e., absences and grade 
averages), which are presented in subsequent chapters, suggest that while for most sites there 
were only small-to-moderate differences between the samples, at one or two sites (including 
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most notably Broward County) these baseline differences were substantial. In fact, no 
statistical approach in this situation can entirely solve the problems stemming from non- 
randomized designs. For absences and grades at all sites, as well a., all survey items 
administered to the 1990-91 student samples at four sites, however, we have elected to report 
gain scores, adjusted for effects of race/ethnicity, age, and grade differences between the 
samples, as these scores are inherently meaningful and provide an unbiased estimate of the 
effect of group membership. (See the discussions of these analyses presented below and in 
Appendix D.) 

Types of outcome measures. This study gathered data on student status — that is, 
graduation, dropout, or continuing enrollment rates — on an annual basis and used these data 
to compare end-of-the-year dropout rates and the average time to dropout among students in 
the treatment and comparison groups. In addition, data were gathered from students and 
teachers describing behaviors that are generally precursors to dropping out — poor academic 
performance, socially disruptive behavior at school, and poor attendance, all of which have 
been linked to school dropout decisions (see, for example, Finn, 1989). While the original 
intent had been to follow all the students selected in 1989-90, the timing of the study and 
changes in the study design, which included the limitation of activities at some sites and the 
addition of new samples at other sites, resulted in different approaches at different sites. 

This report provides and analyzes dropout and retention rates from 1989-90 to 1991-92 
for 10 of the 1 1 middle and high school sites that were originally selected for the evaluation 
(note that one of these sites. New York, was dropped from the evaluation in 1990-91). These 
rates were also analyzed for the four middle and high school sites at which new student 
samples were selected in 1990-91. In these cases, analyses cover the years 1990-91 to 
1991-92. 

At all but three sites, data on daily attendance and yearly grade averages were gathered 
and compared to pre-treatment information for one or more of the years 1989-90, 1990-91, 
or 1991-92. The three sites at which these data were not recorded (or could not be easily 
used) included one project that worked to a large extent with Kindergarten-age studems 
(Los Angeles), the one project at which a longitudinal design was employed (Carbondale), 
and the project that was dropped from the evaluation in 1990-91 (New York). At selected 
projects where the data could be obtained, data on numbers of suspensions in either 1989-90, 
1990-91, or 1991-92 also were compared to baseline totals. Finally, test scores were 
requested from all sites and were collected from student records along with information 
about the test(s) used (e.g., type and level). Norming data were also obtained from 
publishers, with the intention of deriving normal-curve equivalent scores. Due to the variety 
of tests used across the sites and the levels of tests used within sites at specific time-points, 
however, the effort to develop usable achievement measures based on these data was 
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abandoned. In general, the lack of these sorts of achievement measures is unfortunate, since 
the majority of projects reported collecting standardized test data in reading and mathematics 
to assess student progress. For elementary-level projects in particular, the lack of test data is 
especially felt, as grade average data for younger students may be somewhat less 
discriminating (e.g., the range of assignments and grades on assignments may be combined 
within more narrow ranges). 

Data extracted from school records thus included the following: 

• end-of-the-year status information — used to compute dropout rates and dropout 
rates inclusive of students who were expelled or who moved without confirmation of 
re-enrollment (middle and high school students only) 

• percent days absent from school (i.e., total excused and unexcused absences divided 
by total days enrolled and re-scaled to more accurately approximate unexcused 
absences only) 

• average yearly grades 

• number of suspensions (i.e., total in-school and out-of-school suspensions) 
(middle and high school students only) 

It must be emphasized that the typical collection schedules for each of these types of 
data at the sites required several iterations of data gathering and checking by evaluation staff 
involving, usually, school and district sources. In addition, each follow-up data collection 
(i.e., collection of more recent information from a student sample) typically involved 
modifications of the data provided at earlier time-points due, reportedly, to school or district 
updating following the end of the school year. On average, school and district data for such 
indicators as end-of-year student status or total absences tended not to agree perfectly, and 
final values were not derived and made available until several months following the end of 
the school year. 

Survey information was collected from students at all but the one project serving mostly 
Kindergarten-age students (Los Angeles), and the survey returns were sufficient for analyses 
to be carried out at all but four of the sites (Hannibal, Broward County, New York, and 
Memphis). For all the projects selected for the evaluation in fall 1989, survey information 
was collected once from treatment and comparison students — in spring 1990, just aftermost 
of the students had completed one term in their respective SDDAP demonstration projects. 
Due to the lack of baseline data, survey information analyzed for these projects was limited 
to items on school climate and those requesting retrospective evaluations of performance. 
Interpretation of these items require appropriate caution. 

For the four projects at whici< (new) student samples were selected in 1990-91, survey 
data were collected at three points in time: in fall 1990 (baseline survey, prior to involvement 
in the SDDAP project), spring 199 1 (first follow-up survey, just after completion of one term 
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in the project), and Spring/Summer 1992 (second follow-up). In Broward County, where the 
treatment was one semester followed by occasional additional activities during subsequent 
years, semester data were collected for the baseline portion of the 6th-grade year, the 
treatment portion of that year (i.e., the second semester), and for the two 7th-grade semesters 
that followed. 

The data provided by the student surveys included the following items: 

♦ indicators of students' self-esteem (i.e., the Rosenberg Self-Esteem Scale) 

♦ students' attitudes toward school and future education plans 

♦ reports of parental involvement in class/'school activities 
" evaluations of school climate 

♦ perceived performance on school-related activities/tasks 

Although these data must be regarded as less rigorous (and more subject to extraneous 
variation over time) than the data from school records, they nevertheless are useful in 
understanding results from school records. Specifically, the patterns of results from records 
data and surveys may suggest ways in which students' perceptions (e.g., of school climate) 
may have led to greater engagement in schooling (e.g., as measured by grades or attendance). 

At the one site that served Kindergarten-age and first-grade students exclusively (Los 
Angeles), teacher ratings of students' school-related performance were obtained at three 
time-points: Fall 1989 (baseline information). Spring 1990 (first follow-up information), and 
Spring 1991 (second follow-up information). These surveys were particularly important for 
the evaluation, since data from school records provided less than useful indications of student 
performance in this case and student surveys were not administered. Information from the 
teacher surveys covered the following areas: 

♦ classroom citizenship 

♦ classroom participation 

♦ language development 

♦ math/pre-math development 

♦ peer relationships 

♦ parent cooperation 

♦ parent involvement 

At the one site that served re-entry students exclusively (Carbondale), students who left 
the regular school program or the re-entry demonstration since 1989-90 were surveyed with 
specially designed unstructured instruments to learn about their attitudes, personal 
experiences, and plans. A version of this survey was also used in telephone interviews with 
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students who left any of the middle or high school SDDAP demonstrations. 
These surveys inquired into the following: 

• their reasons for leaving 

• their perceptions of improvements that might have caused them to remain enrolled 

• who they spoke to about leaving 

• their reasons for having enrolled in the re-entry program (re-entry site only) 

• their education- and work-related plans 

Finally, because three of the four projects at which new student samples were selected 
in 1990-91 required students to apply for admission (i.e., Broward County, Memphis, and 
the Discovery Program in Denver— or Denver-Discovery), collaborative efforts were made at 
each site to design and administer a brief survey form to treatment and comparison students 
aimed at clarifying the various motivational factors (e.g., peer pressure, parental support) 
affecting students' decisions to participate in special school programs. The greater 
willingness of some students to enroll in projects typically leads to concerns about selection 
bias, which mus: be considered in any attribution of student outcomes to project activities. 
These surveys are inherently limited in that, while the attitudes of program participants are 
formed in light of their actual experience in applying for a real program, non-participant 
attitudes must be based on hypothetical scenarios. Nevertheless, such forms do provide 
information that may begin to suggest ways in which these two groups of students may differ 
with respect to motives— differences that may affect their levels of accomplishment with and 
without special programs. 

The different motivational survey forms included rating scales for the following 

dimensions: 

• parental support 

• peer support 

• ease/difficulty of finding time to participate 

• ease/difficulty of finding transportation to participate 

• perceived usefulness (generally and for the individual) of participation for gaining 
employment 

• perceived usefulness (generally and for the individual) of participation for enhancing 
school performance and/or learning important skills 

• perceived usefulness of participation for helping one to feel more a part of the school 
Sensitivity of outcome measures. Despite their number and diversity across projects, 

the different measures of outcomes used in this study may not be the most accurate indicators 
of the effectiveness or success of the SDDAP projects included in the evaluation. Four 
reasons can be cited for a possible lack of sensitivity of these measures. First, factors other 
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than those studied at specific sites may have affected students' participation in school and 
their identification with the goals, norms, and activities at a school site. Second, data on 
yearly grade averages, percent days absent, and other school-records measurements gathered 
for this study may not provide sufficiendy fine-grained indicators of student improvement 
within a one- or two-year period. Third, policies vary substantially from 'one school, district, 
and state to another. While some uniformity may exist within districts, differences may be 
found, for example, in grading policies and attendance monitoring between schools both 
within and across districts. These differences may affect the straightforward inteip relation of 
results when treatment and comparison samples were drawn from different schools or 
districts. Fourth and perhaps most imponant — the data from school records are affected by 
within-school and within-district differences in recordkeeping procedures. For example, 
school-based repons may not always agree with district summaries, and the lack of district- 
wide policies covering matters such as data-recording procedures can produce significant 
differences in the counts and summaries gathered from individual schools. Although attempts 
have been made to standardize the evaluation data to the extent possible in the analyses 
presented in this report, so that within-site comparisons are based on similarly tabulated data, . 
it is likely that some errors remain. For all these reasons, the findings and interpretations of 
findings presented in subsequent chapters should be regarded as necessarily tentative. 

Overview of analytical approach. Gain-score analyses (or the analysis of follow-up 
minus baseline performance) are used to compare the treatment and control samples on 
measures derived from school records at all the sites and on survey items at those sites where 
new student samples were selected in 1990-91 (i.e., where pre- and post-treatment measures 
were available for analysis). As noted above, these analyses always control for race/ethnicity, 
gender, and age with respect to grade level. The baseline year for all the sites but three that 
were included in the evaluation in 1989-90 was 1988-89. This was the first year of the 
SDDAP funding period, but insufficient progress had been made at most of the sites to 
warrant regarding the year as part of the treatment period. The three of these sites a'c which 
1989-90 cohorts were established and 1988-89 was not used as the baseline year in analysis 
were: Denver-Lake, where the project was fully operational in 1988-89 and, so, 1987-88 
was used as the baseline period; Los Angeles, where, due to the very young ages of the 
students, teacher ratings of students at the beginning of the 1989-90 school year were used as 
baseline measures; and Carbondale, where a longitudinal rather than a matched-pairs design 
was used. For all the four sites at which (new) student samples were selected in 1990-91, 
1989-90 was the baseline period. Comparisons of dropout rates between the student samples 
at the various sites arc made using Fisher's Exact Test, and survival analysis using Cox's 
Proportional Hazard methodology is used to assess differences between these groups in the 
time to actual dropout from school. Like the gain-score analyses presented in subsequent 
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chapters, these analyses of dropout rates and patterns always control for race/ethnicity, 
gender, and age with respect to grade level. (More information concerning these technical 
procedures may be found in Appendix D.) 

The analysis of data from this evaluation has required that attention be paid to several 
needed adjustments. For example, controlling for the effects of selected demographic 
variables in assessing differences between the student samples is an adjustment made to 
minimize the effects of extraneous differences among the groups on outcome variables of 
interest. Similarly, the use of gain-score analysis is intended to minimize the effects of 
baseline differences between the samples on these outcome measures. In addition, 
adjustments were made for less than complete response rates at the sites, unequal 
demographic distributions between student samples at the sites, skewed distributions of 
selected variable values, and imprecise estimates of numbers of total unexcused absences. 
These are briefly described in the following paragraphs and described more fully in 
Appendix D. 

Adjustment for nonresponse— The tables included in Appendices A-C present results 
based both on unadjusted data and on data adjusted for nonresponse (i.e., imputed data). This 
adjustment for nonresponse involves the estimation of missing data values based on what is 
known about the nonrespondents (e.g., race/ethnicity, gender, age with respect to grade, 
whether in treatment or in control group) and the patterns of relationships among.setected 
variables in the database. Data were combined across all the sites in establishing models for 
estimation, with the estimated values for particular sites based on respondent data for only 
those sites. In addition to individual-level data, several subjective indicators describing the 
relative programmatic emphases of the different demonstrations were used in the imputation 
process (e.g., the extent to which projects focused on academics versus vocational 
preparation). 

Adjustment for unequal demographic distributions between treatment and 
comparison groups— To prevent biased estimates due to unequal representation of 
racial/ethnic, age, and gender categories within the samples at a site, the balanced 
representation of these categories within the samples was ensured for purposes of testing 
statistical significance (i.e., if some category was represented in the treatment group at a site 
but not in the comparison group, or vice versa, the cases in that category were removed from 
the analysis). At the same time, the mean values for these variables presented in the tables in 
Appendices A-C have been (re-)weighted to represent the actual demographic distributions 
at the sites. 

Adjustment for skewness in estimates of statistical significance— The sizes of the 
standard deviations for percent days absent, grade averages, and total suspensions were large 
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relative to mean estimates, indicating considerable skevvness in the data. To adjust for this 
skewness, distributions were normalized prior to performing statistical tests. 

Adjustment for total rather than unexcused absence&~For consistency, total 
absences at each of Uie sites were recorded. However, since excused absences (e.g., for 
illness) should be unrelated to most of the efforts of the demonstrations, using total absences 
in analyses may tend to mask tiie effects of tiie treatments. To minimize this problem, Uie 
variable percent days absent was re-scaled prior to analysis to remove as much as possible 
tiie numbers of absences tiiat were likely excused. On average, Uiis re-scaling of total 
absences resulted in about a 2% "decrease" in the estimates of percent days absent for each 
of tiie tiiree grade levels of projects studied (i.e., about 2% of Uie total absences recorded at a 
particular grade level were estimated to be absences that would ordinarily have been excused 
by local school authorities). 

Organization of this Report 

In tiie next chapter, an overview is provided of the in-depth study sites. The results of 
tiie evaluation of tiiese SDDAP demonstration projects through the 1991-92 school year are 
presented in Chapters 3-5 (with data tables presented in Appendices A-C). Chapter 3 
describes the activities and accomplishments of the four demonstrations serving students in 
the elementary grades. Chapter 4 describes tiie services provided and outcomes achieved by 
the five projects serving middle grades students, and Chapter 5 presents the findings for the 
seven high school projects. Each of these chapters begins with an overview, then provides 
descriptions and data for each project, and closes with a summary and interpretation of the 
findings. 

In Chapter 6, we attempt to draw together the findings from the preceding three chapters 
to address the over-arching questions about organizational structures and strategies posed by 
the authorizing legislation. In this chapter also, we present several recommendations related 
to the implementation of dropout prevention programs that are derived from the in-depth 
evaluation and, more generally, from the literature on educating students at risk. Finally, 
Chapter 6 addresses issues related to the sustenance and replication of successful dropout 
prevention programs. Following Chapter 6, Appendices A-C, respectively, provide the data 
tables for tiie elementary, middle, and high school projects. Appendix D provides further 
technical details of the procedures followed, and Appendix E provides copies of all the data- 
gathering instruments. 
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Chapter 2. Overview of the In-Depth Study Sites 



The Differing Contexts of Project Activities 

The in-depth study sites for this evaluation represented a diverse subsample of SDDAP 
grantees. These sites were distributed aaoss the United States, from areas as small as Ethete, 
on the Wind River Indian Reservation in Wyoming, to the large urban centers of Los Angeles 
and New York. (In New York, the demonstration site selected for the in-depth study was a 
community school district in the Bronx.) The projects also aimed to serve student populations 
at varying levels: 

• four targeted elementary school students — Gushing, Oklahoma; Shreveport, 
Louisiana; Hannibal, Missouri; and Los Angeles, California 

• five targeted middle school students— Ponland, Oregon; Broward County, Florida; 
Denver (Lake), Colorado; San Antonio, Texas; and Bronx, New York 

• seven targeted high school and re-entry students— Des Moines, Iowa; Memphis, 
Tennessee; Coleman, Texas; Aiken, South Carolina; Ethete, Wyoming; Denver 
(Discovery), Colorado; and Carbondale, Illinois 

Tables 7-9 provide overviews of selected characteristics of these sites. 



Table 7 

Overview of Elementary School Study Sites 


Site 


Grade 
Levels 
Studied 


Average Size of 
SDDAP Grant 
Per-Year ($) 


Largest 
Race/Group in 
LEA (%) 


LEA Student Population 
(% reported by grantee 
to be at risk) 


Gushing OK 


1-6 


166,428»' 


White (83) 


1.916 
(19.1) 


Shreveport LA 


1-2 


78.726 


black (57) 


52.309 
(11.3) 


Hannibal MO 


K-5 


58.071 


White (92) 


4.319 
(n/a)** 


Los Angeles CA 


K-1 


235.211 


Hispanic (61) 


609.746 
(n/a) 



*Only a portion of these funds was available to the project component included in the evaluation. 
♦•Data not available 
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Table 8 

Overview of Middle School Study Sites 



Isite 


Orade 
Levels 
Studied 


Average Size of 
SDDAP Grant 
Per- Year (i) 


Largest 
Race/Group in 


LEA Student Population 
(% reported by grantee 
10 oe at risk) 


1 Portland OR 


6-8 


419.623* 


White (72) 


51.825 
(66.3) 


H Broward Co. FL 


6-8 


342.001 


White (59) 


148.739 
(7.0) 


i Denver-Lake CO 


6-8 


362.120* 


Hispanic (39) 


58.299 
(39.9) 


San Antonio TX 


6-10 


379.081 


Hispanic (81) 


61.156 
(19.8) 


Bronx NY 


7-9 


403.442 


black (52) 


23.837 
(14.1) 


♦Only a portion of these funds was available to the project component included in the evaluation. 




Table 9 " 
Overview of Secondary School Study Sites 


Site 


Grade 
Levels 
Studied 


Average Size of 
SDDAP Grant 
Per- Year ($) 


Largest 
Race/Group in 
LEA (%) 


LEA Student Population 
(% reported by grantee 
to be at risk) 


Des Moines lA 


9-12 


217.737 


White (82) 


29.760 
(29.4) 


Memphis TN 


10 


379.108* 


black (80) 


105.604 
(5.9) 


Coleman TX 


9-12 


224.834 


White (73) 


1.016 
(15.2) 


Aiken SC 


7-12 


439.717 


White (65) 


23.300 
(n/a)** 


EtheteWY 


7-12 


157.450 


Naiive 

American (100) 


694 
(47.0) 


Denver-Discovery 
CO 


9-12 


362.120* 


Hispanic (39) 


58.299 
(39.9) 


Carbondale IL 


Re-entry 
and 9-12 


119.223 


White (85) 


5,132 
(15.0) 



*OnIy a portion of these funds was available lo the project component included in the evaluation. 
**Data not available 



As is shown on these tables, the projects served students in a range of grades (e.g., from 
K-1 to re-entry students). On average, the amount of SDDAP funding per-project also 
ranged widely— from $58,071 to $439,717 per year. These amounts are somewhat difficult to 
interpret for two reasons, however. On the one hand, the individual awards at a site may have 



supported more than a single local activity. For example, in cases where the evaluation 
focused only on particular components of district-wide dropout prevention programs 
supported by the SDDAP (e.g., Denver-Lake and Denver-Discovery), total SDDAP funding 
for (each of) these components typically amounted to less than one-half of the total grant 
amounts shown — in many cases, much less than half. On the other hand, these grant amounts 
do not reflect the total costs of implementing the projects; cost sharing, which ranged from a 
minimum of 10% to 25% over the course of the period of award, was required of grantees, 
and many projects also received local and state funds as well as grants from other sources. 

The in-depth study sites also were characterized by considerable racial and ethnic 
diversity, as indicated in the third columns of the tables. White students comprised the largest 
groups in the school districts served by eight projects (Gushing, Hannibal, Broward County, 
Portland, Aiken, Carbondale, Coleman, and Des Moines), and, with the exception of the 
Broward County project whose service population was approximately 60% black, white 
students comprised the majority of students served by all these projects. Hispanic students 
comprised the largest groups in districts served by four projects (Los Angeles, Denver-Lake, 
Denver-Discovery, and San Antonio), and in every case but one Hispanic students comprised 
the majority of students served by the projects. (In San Antonio, Hispanic students made up 
slightiy less than one-half of the population served by the project.) black students comprised 
the largest groups in the districts served by three projects (Shreveport, New York, and 
Memphis), and in every case most of the students that were served were black. In Ethete, all 
the students served were members of the Arapahoe and Shoshone tribes. (These data on 
race/ethnicity distributions within the LEAs served were obtained from the Common Core of 
Data.) 

The last columns of each of the tables present two information items: the enrollment 
sizes of the school districts served by the projects or components of projects selected for the 
evaluation (from the Common Core of Data), and, in parentheses next to the district 
enrollments, the estimated percentages of these total populations that^are at risk of dropping 
out, which were reported by the grantees in response to the 1989-90''annual survey of all 
SDDAP grantees. Three projects were located in school districts with fewer than 2,000 
students; four in school districts with 20,000-50,000 students; and three in school districts 
with more than 100,000 students. Los Angeles, with about 610,000 students, was more than 
four times larger than the second largest district, in which the Broward County Project was 
situated. In contrast, the number of students enrolled in Ethete was little more than .1 1% of 
the number enrolled in Los Angeles. 

In terms of the estimated needs of these populations, the reported district percentages of 
students at risk ranged from 5.9% in Memphis to 66.3% in Portland. (Data on percentages of 
students at risk were not reported for Los Angeles, Hannibal, and Aiken in 1989-90, as 
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indicated on the table.) In interpreting these estimates of at-risk population sizes, it is 
important to emphasize that they are in fact estimates reported by the projects and may have 
been developed through consideration of various data sources and using various criteria. 
Although a general definition was provided of at-risknesSi respondents may have used more 
or less specialized definitions. For these reasons, the estimates are best viewed as illustrative 
examples of the ways in which districts report student needs. 

Project Goals and General Approaches 

Projects did not receive specific federal direction as to how they were to allocate the 
monies from their SDDAP awards. As suggested previously, funds could be combined with 
funds from other sources or, alternatively, used as a single source (with necesary matching 
funds) in support of dropout prevention activities. Given this flexibility, the 16 projects or 
project components involved in the in-depth evaluation reflected a range of goals and 
approaches for improving the educational experiences and outcomes of children and youth at 
risk of educational failure. (It should be emphasized that several projects offered various 
services to students at risk across the grade levels in one or more special programs, but the 
in-depth evaluation of necessity focused on specific program components in these cases. For 
example, the Gushing, Oklahoma, project served students in grades K-12, but we examined 
the project's services to elementary. school students only in the evaluation.) Table 10 
describes this diversity, summarizing the purposes of the grants, the organizational structures 
employed by the projects, the key services that were provided by the projects to students, and 
the key intermediate goals the projects were attempting to accomplish. 

As Table 10 shows, 10 of the grantees expanded projects begun prior to receipt of 
SDDAP funds (e.g., the Carbondale project had been in operation since 1970). Several of the 
projects also implemented some form of alternative organizational structure on the theory 
that children and youth who were at risk educationally would be more likely to thrive in 
environments that differ from the traditional school organization. Three were alternative 
schools, located in different facilities from the district's regular schools. Two were schools- 
within-schools. Four provided services outside the regular school day, either after school or, 
in one instance, on Saturday. Two projects in the study did not target at-risk students 
specifically: one was a school-wide project (Hannibal) and one served all the students at a 
particular grade level in the school (New York). Three projects implemented a cluster 
approach to enrichment, and three used a fairly standard pull-out model. 

Nearly all projects provided basic skills instruction intended to improve the academic 
performance of participants. Five implemented academic enrichment, and three others 
included employment-related activities among their services. Nine provided individual, 
Sroup, or family counseling to participants, but only five included an active parent 
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Overview of Proiect Aims and AoDroschcs 






Number of 




Prniprts 




D Purpose of the grant 


in 


1 Expand an existing project 


1 Establish a new proiect 


o 


1 Organizational structure 




1 Alter schooi/oaturaay/extenuea oay 


4 


1 Pull-out 


J 


1 Cluster/enrichment 


J 


H Alternative school 


1 

J 


B ocnooi-wiinin-a-scnool 


2 


School-wide project 


1 
1 


Key services 


12 


AcaQemic reuieoiauon/vjcL/ prcpaiauun 


Counseling 


9 


Academic enncnmeni/innovauve insuiu^uun 


5 


Parent component 


5 


Emplovability-related activities 


3 


Key intermediate goals (related to drop-out prevention) 


14 


Improvement in academic achievement 


Improvement in self-esteem 


5 


Increase in parental involvement 


5 


Career preparation 


3 



component, a relatively small number given the research suggesting the importance of 
parental involvement activities in improving the educational experiences and outcomes of all 
students, particularly those considered at risk. 

Predictably, all the projects listed dropout prevention as a key project goal, although 
only two sites— Coleman, Texas, and Des Moines, Iowa— set specific numerical goals 
related to preventing dropout from school. Other objectives articulated by the projects as 
important in achieving dropout prevention included improvement in academic achievement 
(14 projects), improvement in participants' self-esteem (five projects), increased parental 
involvement (five projects), and career preparation (three projects). 

Projects Serving Elementary School Students 

The grade levels of students served by projects at the elementary level varied, with 
Gushing and Hannibal serving K-5 and K-6 students, respectively, and Shreveport and Los 
Angeles serving students in grades K-1 and 1-2, respectively. In Gushing and in Los 
Angeles, students recruited for the projects were identified by teachers (i.e., the selection was 
non-random); in Shreveport, principals and counselors made these selections, with priority 
given to those students who had been retained or were achieving below grade level in 
reading. In Hannibal, all students at Eugene Field Elementary School participated in 

O 43 47 

ERIC 



accelerated learning activities. The four projects were equally divided in relative emphasis on 
academic versus social concerns. All had strong parent components, and this strategy was 
generally more successfully implemented by these projects at this level than by the projects 
targeting middle and high school students. 

Participants 

One project, Hannibal, was a school-wide initiative, and thus included all students in 
grades K-6. The three remaining projects selected students for participation. In Los Angeles, 
an average of three Kindergarten and Ist-grade students per class at each of 24 schools were 
identified by their teacher as needing extra help in adjusting to school. Criteria for selecting 
these students included aggressiveness, extreme shyness, lack of motivation, hyperactivity, 
and low achievement levels. In Shreveport, 1st- and 2nd-graders were selected by principals 
and guidance counselors from all students who were eligible for Chapter 1, with preference 
given to the lowest achievers and those retained in grade. In Cushing (i.e., in the component 
of the Cushing project studied by this evaluation), students were identified as being at risk as 
a result of poor attendance, skill level, and test scores. 

Project Goals 

All four elementary projects emphasized dropout prevention in their stated goals. All 
sought to provide primary students with early successes in school to develop their 
commitments to school and to enhance their potential for academic success. Additionally, the 
specific objective of Hannibal's school-wide project was to bring all students up to grade 
level by completion of the 6th grade. 

Organizational Location, Governance, and Staffing Issues 

The Hannibal, Shreveport, and Los Angeles projects used SDDAP funds to expand 
existing projects. The Cushing project began with the award of funds. Although all four 
projects operated in regular public schools, only the Hannibal project was administered by 
the school itself. The Shreveport project was operated by a community-based organization, 
Los Angeles by the district's School Mental Health Center, and Cushing by the Oklahoma 
Child Demonstration Service Center. 

The elementary projects placed a great deal of emphasis on recruiting and training 
skilled staff. In Shreveport, certified teachers were selected to run an after-school program by 
their building principals. Staff development for the teachers and aides in Cushing and the 
volunteers and regular teaching staff in Los Angeles was a high priority. In Hannibal, the 
SDDAP award paid for substitutes during training days for teachers; at least eight full days 
each year were devoted to learning and working through the accelerated schools process. 
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Strategies Geared Toward Student Change 

Three of the four projects provided basic skills reinediation. Shreveport's was an after- 
school project that used the Peabody Language Development system to develop children's 
early literacy skills. In addition, Shreveport students had an hour a day of enrichment 
activities provided by community members (e.g., artists, musicians) and received help with 
their homework from paraprofessionals. Gushing had both an after-school tutoring program 
and a summer school to help students keep up academically, supplemented by both group 
and individual counseling. In Los Angeles, college students, community members, and 
parents formed the volunteer corps that provided psycho-social and educational support to 
help identified students adjust to the regular classroom. 

In Hannibal, the entire curriculum was reshaped to the extent possible to shift away 
from a traditional remedial pull-out model, and the school adopted an accelerated schools 
model. Teachers worked to revise instruction to provide greater challenges to all students and 
more opportunities to practice advanced skills. For example. Project Read, a model adopted 
by many accelerated schools, is a whole-language approach in which the goal is to have 
students use language as a tool for thinking and coinmunication in all classes — not just 
reading. The Intervention-Before-Retention component of the Hannibal project identified 
students with problems and worked on individual solutions with parents and staffing teams. 
There was also an after-school tutoring program made available to students at a nominal cost 
to families. 

Family Involvement 

Parents in Hannibal were involved in individual student evaluation, as described above, 
along with representation on governance committees. They also completed a needs 
assessment survey that was used to develop the annual school plan, and they received 
learning materials and newsletters from the school on a regular basis. The goal of these 
activities was to engender active long-term participation in the educational process among 
parents. 

Parent participation was a major goal in Los Angeles as well. In 1989-90, between 70 
and 80 parent discussion groups took place to address issues that v/ere raised by the parents 
themselves. These groups were conducted separately for both English- and Spanish-speaking 
parents, with the result that participation rates were high for both groups. Parents were also 
involved as volunteer tutors. In Shreveport, parent volunteer time was a requirement, and 
parents had to agree to pick up their children at the project every day, thereby pemiitting 
. maximum contact with teachers. The project also used a computer-driven telephone system 
with taped project updates and "Tips for Parents." In Gushing, parents could help present 
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structured classroom activities to develop the children's self-esteem, and they also 
volunteered to help out in the classroom as "Celebrity Readers." 

Changes in School Structure or Schedule 

Both Shreveport and Gushing extended the school day for after-school programs, while 
Los Angeles strengthened instruction through the use trained adult volunteers in classrooms. 
While after-school projects do not necessarily qualify as structural changes, in the case of the 
Shreveport project, this component extended the school day for teachers, principals, and 
other staff, as well as students. For these participants, school has changed in an important 
way and can be regarded as a structural change. 

The Hannibal project chose a site-based management approach and completely 
reorganized the structure of school governance. Committees, or "cadres," with both teachers 
and parent representatives, were organized and made decisions on a variety of school issues: 
discipline and self-esteem, parent involvement, the extended-day program, and curriculum 
strategies. When cadres made recommendations, the school "steering committee," composed 
of representatives of the individual cadres, the administration and office and support staff, 
and a parent representative, voted on them. While the principal retained the final word, he 
generally supported the positions reached by the steering committee. 

Comprehensiveness and Coherence of Project Services 

Table 1 1 provides a summary of the service designs implemented by each of the four 
projects targeting services to elementary students. In general, a project was detei-mined to 
include a service element if the element was central to the project's design of services. For 
example^ a project's "parent involvement element" must have involved a set of activities and 
interactions with parents that not only intended, but also achieved, active and meaningful 
involvement of the families of panicipants. Similarly, "counseling" was defined as formal, 
routine provision of counseling services by a trained counselor, rather than occasional 
counseling by teachers or principals. 

As noted on Table 1 1, all four projects emphasized development of academic skills and 
parent involvement. In Gushing and Los Angeles, counseling seivices were also provided to 
students, and in Los Angeles, counseling on how to work with particular students was 
provided to teachers as well by trained mental health professionals. Counselors were 
assigned to the Gushing demonstration; in contrast, the counseling of students in Los Angeles 
was 
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Table 11 

Summary of Project Service Designs for Elementary School Projects 


Projects 


Gushing 


Shreveport 


Hannibal 


L.OS Angeics 


Service Element 










Academics 


X 


X* 


X* 


X 


Parent Involvement 


X 


X 


X 


X 


Counseling 


X 






X 


1 Attendance Monitoring 


X 








1 Support Services 






X 


X 


1 Linkages 


X 






X 



* Indicates accelerated, or enriched, academic instruction. 



Key: 

Academics — Services may include remediation in basic skills or emphasis on particular topics, as well as 
accelerated or enriched instructional opportunities 

Parent Involvement— Services may include parent training sessions, as well as activities aimed at ensuring 
parent knowledge and support for project efforts 

Counseling— Services may include individual or group counseling aimed at a wide-range of youth-rclatcd 
problems (e.g., problems with parents, stress) 

Attendance Monitoring— Services include direct monitoring of student attendance on a daily basis and 
may include active follow-up of students who are absent 

Support Services — Services may include provision of health care for students, as well as other services 
that are intended to make it easier for students (or prospective students) to attend class 

Linkages— Services include referrals to other community agencies (e.g., for housing or income assistance) 

most frequently provided by volunteers (high school and college students, parents, and other 
community members) who received special training and provided one-on-one attention to 
identified kindergarten and Ist-grade students in their regular classrooms. Student support 
services were available in Hannibal, where they were typically arranged by teacher and 
parent teams working to improve students' chances for achievement, and in Los Angeles, 
where teachers or volunteers may have provided the support. Finally, the Gushing and Los 
Angeles demonstrations established linkages with other community agencies that were called 
upon to assist students with special problems. 

Two projects — Hannibal and Los Angeles — provided services within the regular school 
day, while Gushing and Shreveport extended participants' school days by implementing 
after-school (and, in Gushing, summer) activities. Hannibal also offered after-school 
activities for students. The Gushing and Los Angeles projects offered more types of services 
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overall — in five of the six categories. However, only one of the projects in this group 
represents a major structural change in children's experience of school. As noted previously, 
the Hannibal project implemented the accelerated schools model, in which students at risk, 
along with their peers, experience an altered school environment, in terms of schedule, 
curriculum, and other factors. 

Projects Serving Middle School Students 

Four of the five SDDAP middle-school demonstrations participating in this evaluation 
chose to address both students* academic and social participation in school. The remaining 
project (Portland) provided an elaborate support services networic to address non-academic 
problems that interfere with school experiences and could eventually cause students to drop 
out. 

Participants 

Most of the middle school projects in the in-depth evaluation followed a traditional 
targeting model for at-risk interventions: students were identified as being in need of support 
services or academic remediation. Only one of the programs, Denver-Lake, did not explicitly 
select students based on their perceived risk of dropping out; to facilitate program evaluation 
in a setting that served predominately youth at risk, an entire "core" (or "team") of students, 
chosen at random from the middle school's 6th-grade population, was enrolled in the 
demonstration program. This program consisted of a restructured schedule, staffing 
configuration, and system of rules. 

The other four projects relied upon combinations of referrals and recruitments to create 
pools of eligible students. In Portland, where project staff reported they served a highly 
disadvantaged student body, all students were able to take advantage of the range of support 
services that constituted the treatment. While some students volunteered for counseling with 
the student services specialists and many others interacted with these specialists outside of 
formal counseling sessions, most participants in these sessions were referred by teachers as 
having a variety of problems (such as family violence, drug dependence) that interfered with 
success in school. 

In New York, counselors used an informal screening system to identify students in need 
of academic remediation, although all students were eligible for project services. In Broward 
County, students who were low-achievers, over-age for their grade, or exhibited behavioral 
problems were identified and invited to participate. For students to be eligible, however, their 
parents had to agree to attend weekly parent education seminars. Students were enrolled in 
the program on a t'lrst-come first-served basis; that is, the first 30 applicants per sen-iester at 
each site were accepted. San Antonio followed perhaps the most elaborate selection 
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procedure: the eligibility determination procedure involved a detailed point system based on 
socioeconomic status, non-mastery of the state test of minimum education requirements, 
achievement test scores at two years below grade level, and other factors. Once the pool of 
eligible students was identified, students were recommended for participation by teachers and 
administrators, and parents were then contacted. 

Project Goals 

The five middle school projects all stated similar goals. The most important long-term 
goal for all was keeping students in school, and the various projects cited intermediate 
objectives that demonstrated a common understanding of the problems interfering with 
school success. Four of the five projects specifically aimed simultaneously to improve 
academic achievement and to enhance self-esteem by providing a supportive environment. 
Discussions with project directors and staff made it clear that they were keenly aware of the 
need both to create stronger emotional links to the school and to increase the skills necessary 
to succeed academically. 

The fifth project, Ponland, did not address academic issues directly, but instead focused 
on environmental and psychological factors that can interfere with student adjustments to 
middle school. Because they were dealing with a highly at-risk population, the decision was 
made to concentrate effons on problems such as drug abuse and family violence that make 
improved academic achievement extremely unlikely. The project staff believed that 
stabilizing the students — i.e., giving them "anchors" to stay in school — was a prerequisite to 
school success. 

Organizational Location, Governance, and Staffing Issues 

All five middle school projects were run by their school districts, and in every case the 
principals at the sites had considerable input into the hiring of key staff. In Ponland, for 
example, staff were selected by the principal and district together and were chosen for their 
specific experience in dealing with out-of-school problems of youth at risk. In Broward 
County, the principals of each of the three sites engaged in replicating the Model School 
Adjustment Program (the locally developed dropout prevention initiative) were responsible 
for all staffing decisions concerning the project. 

According to local reports, the staffing arrangements were judged as generally 
appropriate. At two sites, there were signs of resentment from the non-project staff at what 
was perceived to be special treatment of the project staff (reduced class size or fewer or 
greater responsibilities). For the most part, however, project staff relationships with other 
faculty seemed satisfactory. 
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Strategies Geared Toward Student Change 

While the five projects differed in their amounts of focus on academic concerns, an 
emphasis on counseling was common to all. In all cases, counselors were hired specifically 
for the projects. In New Yoric, the counselors were consulted on an ad hoc basis; in Denver 
the counselor met regularly with groups of students. In San Antonio, students were referred 
to the counselors by project teachers, and counselors also conducted regular group sessions. 
In Broward County, counseling took a number of forms: individual, group, peer-directed, and 
family oriented. 

The Portland project, as noted previously, offered a complex network of counseling 
services. The student services specialist acted as a case manager and counselor, supervising 
and organizing the other components as well as participating in crisis intervention. The drug 
and alcohol counselor provided various types of group and individual sessions for children of 
substance abusers, children who themselves were drug users, returnees from treatment 
programs, and nonusers concerned about drug and alcohol issues. The drug and alcohol 
counselor also provided awareness sessions in regular classrooms and trained school faculty 
on substance abuse issues. There was also a support group for victims of sexual abuse. 

About 30 students in the Portland program were referred for tutorial help in math and 
reading skills, but this was not the central focus of the project. Similarly, in Broward County, 
peer academic tutoring was an important but not central component of the project design. In 
the remaining three projects, improving academic achievement comprised the core of the 
programs. 

In New York, students were tutored after school, and a structured incentive system 
provided a variety of rewards for academic success. In Denver-Lake, the project "team" had 
a schedule that allowed for more class time in core subjects and for more intensive 
exploration of particular topics. In San Antonio, project students had one- or two-hour daily 
computer-assisted basic skills sessions designed to help them meet the requirements of the 
Texas competency tests. 

Family Involvement 

While all of the project staff emphasized the need for parent involvement in school 
activities, the projects reported varying degrees of success at obtaining parent participation. 
In New York, newsletters were sent home and contacts were made with the parents of the 
students involved in counseling. In Portland and San Antonio, there were parent discussion 
groups and seminars. Denver-Lake had a parent support group, and the counselor with the 
program visited the homes of students having problems. 

Broward County had perhaps the strongest parent component. Before students could be 
selected to participate in the project, parents had to sign an agreement to participate in the 
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once-a-week parent education seminars. In addition, the project offered counseling, drug 
education, and referral services to families as well as students. Because involvement was 
mandatory, 100% of the Broward parents tended to participate, although excuses were 
sometimes given for unavoidable absences. If a parent dropped out of the parent education 
seminars, students were removed from the project. 

Changes in School Structure or Schedule 

San Antonio, Denver-Lake, New York, and Broward County all used SDDAP funds to 
add teachers and reduce class size. In Portland, the student ser/ices specialists helped 
students resolve behavior problems before suspensions occurred. All of these individual 
adjustments in "normal" school operations changed the school experience. 

The Denver-Lake project made a more comprehensive effort at restructuring the school 
environment for the members of the team. In addition to adding an extra teacher to reduce the 
student/teacher ratio, they developed alternative schedules, grading systems, and discipline 
procedures. Rather than taking five academic subjects daily, students took four core subjects 
on a rotating basis, which allowed for longer class periods. Enrichment classes in the 
afternoon were devoted to in-depth study of particular areas of interest. Students did not 
receive failing grades during the first two years of the project; if their performance was rated 
below "C," they were given time to make up their work during the enrichment class time. (In 
the third year, students were placed on a "regular" grading system to prepare them for high 
school.) Disciplinary issues were handled entirely within the team by the teachers, rather than 
going through an assistant principal or another school administiator. Finally, Denver-Lake 
project students were exposed to the team-teaching arrangement throughout the three middle 
school years. 

Comprehensiveness and Coherence of Project Services 

Each of these demonstrations operated within the traditional school setting, and a 
summary of their project service designs is presented in Table 12. As noted in Table 12, 
Broward County's Model School Adjustment Project (MSAP) aimed to address multiple 
needs of students at risk through provision of academic remediation (with peer tutore and 
extra periods of instructional support); an intensive parent education component; group 
counseling for all participants, along with individual and family counseling as required; 
referral to external agencies for supportive services; and proactive attendance monitoring and 
intervention. Additionally, the project included longer-term student follow-up through 
enrollment in high school. Thus, while the project's design specified student participation for 
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Table 12 

Summary of Project Service Designs for Middle School Projects 


Projects 


Portland 


Broward 
County 


Denver- 
Lake 


San Antonio 


Service Element 










Academics 




X 


X* 


X 


Parent Involvement 


X 


X 


X 


X 


Counseling 


X 


X 


X 


X 


Attendance Monitoring 


X 


X 






1 Support Services 


X 


X 






i Linkages 











* Indicates accelerated, or enriched, academic instruction. 



Key: 

Academics — Services may include remediation in basic skills or emphasis on particular topics, as well as 
accelerated or enriched instructional opportunities 

Parent Involvement — Services may include parent training sessions, as well as activities aimed at ensuring 
parent knowledge and support for project efforts 

Counseling — Services may include individual or group counseling aimed at a wide-range of youth-reiated 
problems (e.g., problems with parents, stress) 

Career Preparation — Services may include career exploration, career awareness, or work-experience, as 
well as vocational training 

Attendance Monitoring — Services include direct monitoring of student attendance on a daily basis and 
may include active foUow-up of students who are absent 

Support Services — Services may include health care or parent training classes for students, as well as 
other services that are intended to make it easier for students (or prospective students) to attend class 

Linkages — Services include referrals to other community agencies (e.g., for housing or income assistance) 

only one semester, the exposure during that semester, particularly when parent exposure is 
added, was relatively high. Further, the commitment to longer-term follow-up, even if actual 
contacts were infrequent, did appear to send a message to students concerning the importance 
attached to their progress through middle school. Similarly, in its efforts to restructure 
students' school experiences, the Denver-Lake project emphasized maintenance of teacher- 
teams across the three middle-school grades, changes in the way subjects are scheduled and 
taught, teacher control of student discipline, an altered grading system, and elimination of 
tracking (or even identification) of students according to ability. For these students, then, the 
project addressed their entire educational environment. 
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While most of the middle-school projects included academic instruction (primarily basic 
skills in Broward County and San Antonio, and both basic skills and academic enrichment in 
Denver-Lake and New York), the Portland project omitted this component from its service 
design. Targeting students believed to be at highest risk, the project aimed to address 
personal and family support needs of students on the theory that academic progress is 
unlikely unless (and until) serious personal issues can be resolved. 

Projects Serving High School Students 

As might be expected given the ages of the youth they served, the seven high-school 
level projects differed in important respects from interventions designed for elementary or 
middle school students. Some of these differences were based on the recognition that 
important non-school factors can affect older youths' motivation— or ability— to remain in 
school. For example, three sites— Memphis, Des Moines, and Coleman— placed considerable 
emphasis on a variety of employment-related services, using the promise of jobs as an 
incentive to keep participants in school. Others offered a variety of supportive services (e.g., 
child care, case management, referral to social services) that students might need in order to 
continue their education. Finally, all the sites, perhaps recognizing that by high school youth 
are often seriously alienated from the regular school environment, provided at least some 
services in alternative settings. 

Participants 

For project participation, the seven high school projects in the in-depth evaluation 
targeted students considered to be at risk of dropping out, although the projects varied 
somewhat in the ages of youth selected for services and in the criteria used for determining 
eligibility. To identify students for participation, most used some combination of the standard 
factors associated with risk: low achievement (i.e., low grades and credits toward 
graduation), poor attendance, cusciplinary or legal problems, high mobility, over-age for 
grade, and poverty (e.g., eligibility for free lunch or other measures of socioeconomic status). 
Generally, eligible students were identified and referred by school counselors, teachers, or 
administrators. The Denver-Discovery project, for example, attempted to identify incoming 
9th-grade students for project participation on the basis of these types of indicators and 
recommendations. 

Several of the projects selected students based on additional or less-standard criteria. 
The Memphis project, for example, targeted highly at-risk lOth-graders and used full-time 
summer and part-time school-year jobs as an incentive. Because the employment component 
of the project was funded through the Job Training Partnership Act (JTPA), students had to 
be JTPA-eligible in addition to meeting the other criteria for project eligibility. The Des 
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Moines project, which also used work experience as an incentive, largely limited eligibility 
to students with attendance problems. The director's view was that poor attendance, more 
than low achievement or other factors, is the chief indicator that students have given up on 
school. Ethete, Wyoming, the project that served Native American youth, was open to all 
students on the assumption that all Native American youth are at risk of dropping out of 
school. Finally, several projects used situational or idiosyncratic factors as one component in 
their identification of youth for participation: the Aiken, South Carolina, project targeted 
youth in foster care or from single-parent families, while Coleman targeted youth who have 
failed the state competency exams. 

The re-entry project in Carbondale, Illinois, was open to all dropouts and to enrolled 
students who were close to failure in the city and in the two surrounding districts that had 
implemented arrangements for participation. Because of this project's open entry-open exit 
policy, participants could come and go, often entering soon after they had dropped out, then 
attending sporadically for a period, and then often returning and persisting through 
completion of the GED or credits they needed to obtain a diploma from their home high 
schools. According to the project director, several factors affected youths' attendance 
patterns. Those who were close to graduation (i.e., required only two or three credits when 
they dropped out) were likely to attend regularly and finish quickly. Younger participants 
(often those lacking many credits) typically attended only irregularly until they reached 18 
years, the legal age in the state for GED eligibility. They often then entered and quickly 
completed the GED. 

Project Goals 

Expectably, the overriding goal of all seven projects was to keep youth in school until 
they had earned a high school diploma (or GED). To respond to the emerging needs and 
interests of youth in high school, most projects also specified intermediate goals that 
supported the major dropout prevention focus of project activities. Recognizing both that 
high school-aged youth who are at risk of failure are often turned off by the traditional 
organization of school and that these youth are more likely to persist if they can improve 
their basic skills, all projects aimed to improve participants' academic achievement, typically 
through some form of alternative instructional strategy or organization. A key objective of 
the Aiken and Coleman alternative schools, for example, was to bring participants' academic 
achievement and school affiliation to a level that would enable them to return to and succeed 
in their home high schools within a semester or a year. 

Three of these high school projects articulated career preparation (for Memphis and Des 
Moines, this meant employability skills; for Coleman, vocational training) as an important 
objective. The notion was that some students, based on personal circumstances, must earn 
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money in order to stay in school. Additionally, the self-esteem associated with succeeding on 
a job and having money to spend like other students was thought to have the ancillary benefit 
of improved school affiliation. Finally, career preparation was considered in itself to be an 
important benefit to youth who will enter the labor market following high school (even if 
they do not complete school). 

Some of the projects focused on environmental and psychological factors that interfered 
with youths' educational progress. The notion here was that addressing the personal and 
family problems of youth at risk might well help to solve their academic difficulties. In this 
context, the objectives associated with students* personal circumstances involved reduction 
in family dysfunction or improvements in participants' living, health, and mental health 
status. Only one of the high school SDDAP projects (i.e., Denver-Discovery) stated parent 
involvement as an important goal and organized activities specifically to involve parents in 
their children's school experiences. 

Four of the high school demonstration sites paid particular attention to creating more 
positive and more personalized school environments, and three of these projects were 
organized in alternative school facilities (i.e., Aiken, Coleman, and Carbondale). In Aiken, 
where students attended a high school in the district set aside for full implementation of the 
Diversified Educational Experiences Program (DEEP), the aim was to create a self- 
governing, project-focused student body that interacted closely in and outside school with the 
principal and teachers. In Coleman, where students from several districts were sent to one 
high school that offered remedial classes, counseling services, and paid-work opportunities 
on campus, the aim was to provide the students with a family context. And in Carbondale, 
the small numbers of students present at any one time allowed teachers and students to 
interact informally. At Denver-Discovery, where an important goal was to ease the transition 
from middle school to 9th-grade, a supponive school climate was promoted through team- 
taught classes and meetings within the regular high school. 

In contrast to these sites, the other high school demonstrations worried a bit less about 
the climates(s) at the students' home schools. In Ethete, for example. Native American tutors 
were added to the regular school-day experience for only one class period. Similariy in Des 
Moines, the students in the combined New Horizons/School-Within-a-School (NH/SWS) 
demonstration did attend smaller classes during selected periods of their days at school, but, 
beyond greater personal attention from their teachers and opponunities for career/vocational 
exploration activities, no efforts were made to change the climate of the school generally or 
to build particular ties to the NH/SWS project. Finally, in Memphis, students attended 
Saturday day-long activities outside their regular school environments, so that little or no 
change i.i their home school climates was expected to result from SDDAP participation. 
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Organizational Location, Governance, and Staffing Issues 

Four of the sites used SDDAP funding to implement a new project, while two expanded 
existing projects (one has been operating since 1970). The seventh project used funds to 
extend an existing work experience component to students enrolled in another of the 
district's projects operated under a comprehensive system of dropout prevention and re-entry 
services. 

Three of the projects CDenver-Discovery, Des Moines, and Memphis) were 
implemented as part of large, multi-component dropout prevention programs operated by the 
school districts. The Denver-Discovery project was begun by the Denver Public Schools in 

1989- 90 and, to an extent, was modeled after the Denver-Lake team-teaching program. The 
project director at North High School in Denver reported to his school principal and, by 

1990- 91, to an assistant principal at the school as well, while also reporting to the district's 
coordinator of dropout prevention programs. The Des Moines project was located in an 
umbrella office created in 1968 to provide work experience and community service 
opportunities to high school students in the Des Moines public schools. The office director 
reported to the district's director of secondary education, although the program operated 
largely autonomously (less than half of the funds were supplied by the district). The director 
was responsible for hiring staff, who were required to have teaching or counseling experience 
or close links with the city's employers, depending on their job responsibilities. The 
Memphis collaborative, supported in part with foundation funds, was a cooperative effort of 
the school district, the business community, the Private Industry Council, a private 
foundation, and other supporting organizations. The collaborative operated projects for 
middle school students, 10th- and 1 Ith-graders, and juvenile offenders in addition to the 
project supported by SDDAP. Project teachers were required to be certified, career-ladder 
teachers who volunteered based on their interests in and experience in teaching youth at risk. 

The three alternative-school sites differed considerably in their organization and 
governance. The project in Aiken was managed at the district level by a team comprised of 
the associate superintendent of instructional services, director of grants management, and 
executive director of special programs. While this group was responsible for planning and 
policy, the school's principal had substantial autonomy in decision-making concerning the 
hiring of staff and the determination of the types of approaches to be taken in serving 
participants. In Carbondale, the project director and one or two key staff persons who had 
been with the project for many years made all the day-to-day decisions. While supervisory 
authority for the project rested with the coordinator of vocational programs at Carbondale 
High School, his role in decisions concerning the project was limited. In Coleman, the 
alternative high school was operated cooperatively by several small, independent school 
districts, with an advisory board made up of the superintendents of the seven districts. Again, 
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day-to-day decision making was the province of the school's principal and key staff 
members, who also had major responsibility for selecting staff. In addition, (even) the 
advisory board meetings were typically called by the alternative school principal. 

Some of the projects experienced difficulty with staff turnover. One of the alternative 
schools, for example, which had carefully screened staff to ensure their appropriateness for 
working with the recruited students, was forced to dismiss two teachers because they were 
unable to follow the school's philosophy in working with these students. Another project lost 
staff because of the uncertainty of future sources of funding. In that project, the project 
directors and instructional staff were not categorized as district employees; consequently, 
their continued employment depended on "soft" money, and several accepted permanent 
teachinf, or administrative positions. Finally, at one site where teachers had been appointed to 
the pro.ect, there was evidence that teachers may have left because they had not received the 
appropriate orientation and training for the work. 

Strategies Geared Toward Student Change 

These seven high school projects reflected the differentiation that distinguishes high 
school from elementary and, for the most part, middle school. They also reflected notions 
about what could be expected to work with high school-aged youth, which in turn influenced 
the types of instructional and other services that the projects undertook. Perhaps for these 
reasons, and because there is less consensus about what does work with older youth who are 
at risk of dropping out of school, the projects tended to differ from one another more than did 
those demonstrations at the other grade levels. 

Academic components. All the projects placed at least some emphasis on academic 
preparation, and for all seven, this emphasis included basic skills remediation. The three 
alternative schools provided the full academic program that high school students need to 
graduate. One made use of reduced class size and individualized and computerized 
instruction in an attempt to bring students up to age/grade level as quickly as possible. 
Another placed strong emphasis on an alternative instructional system that provided 
academic training in an environment that helped students learn to cope with the system. The 
third project used flexible hours, student contracts, and vans to transport students to and from 
the site as means of making school more attractive to students and amenable to their personal 
and work schedules. According to the project's director and staff, many students were able to 
complete full-year courses in science, math, and other subjects in six to eight weeks, in part 
because they were removed from the stress that, for them, characterized the regular school 
environment and often limited their ability to perform to the level of their classmates. All of 
these schools intended that some or all participants would return to their home schools to 
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complete their high school education, although students who were unwilling or unable to do 
so could remain within the alternative settings through graduation. 

Des Moines, a school-within-a-school/work experience program, provided 
individualized instruction in math, English, and social studies in small classes (from 1 to 15 
students per class). This reduced class size was intended to enable teachers to become 
sufficiently familiar with their students* academic needs to tailor instruction and help them 
improve their achievement. In Denver-Discovery, selected 9th-grade students were assigned 
to work with teacher teams in core subjects, and these teams met on a regular basis with the 
students and their parents at the school to review progress and study habits. Special 
ceremonies were scheduled, with students' parents in attendance, to celebrate various types of 
course-related accomplishments. 

The Memphis Saturday School included academic remediation and enrichment as one of 
a number of activities that filled each of 20 Saturdays for participants. Teachers (certified in 
English or mathematics) provided instruction in test-taking strategies, problem- solving, 
. motivation, and other topics in addition to instruction in basic skills. Each teacher worked 
with about 12 students in each class, with the objective to foster good student-adult 
interactions in addition to the focus on skill building. Since students were drawn from a 
number of the city's high schools and represented a variety of achievement levels, academic 
instruction was not panicularly well integrated into the students' regular high school courses, 
although specific assistance with the regular school work was not the intent of the project. 

Academic support took the form of tutoring in the Ethete project. Tutors were adults, 
and students who appeared to need assistance were offered help on an apparently ad hoc 
basis. Typically, a student received tutorial services for a week or so, and then the tutor 
moved on to another student. Areas of assistance were identified by the student or by a 
teacher. 

Vocational components. Three of the seven projects placed specific focus on 
vocational preparation, although only one provided vocational education within the 
(alternative) school setting. The main focus of the Memphis project was to provide 
participants with instruction in employability skills, and, perhaps most important, activities 
whose intent was to improve motivation and self-esteem, which can contribute to career 
preparedness. Students who attended regularly throughout the year and made acceptable 
progress were promised the reward of a summer job, and several participants commented that 
what they liked best about the project was the job. 

Similarly, a key component (the component supponed with SDDAP funds) in the Des 
Moines project was paid-work experience for panicipants. Again, the job was an incentive; 
students were not allowed to get paid for work on days they missed school. In addition to the 
job itself, project staff provided instruction and counseling in employability skills and career 
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awareness. These staff also were responsible for visiting job sites to monitor student 
performance and for conducting home visits to consult with parents on their child's progress. 

A key goal of the Coleman alternative school was to provide students with a marketable 
skill. Thus, the project provided a range of vocational education activities for students. 
Freshmen enrolled in an exploratory course, and other students had a number of programs to 
choose from, including business and office skills, small engine repair, auto shop and welding, 
printing, construction trades, and commercial food production. Some work experience was 
available, principally, through JTPA programs operating in the summer. During the first two 
years, some of this experience was provided on the school campus, when students were 
"hired" to assist with food service and school maintenance functions. Project staff believed 
that the vocational component of the school was important in keeping participants in school: 
vocational preparation was seen to provide students with expectations of achieving an 
immediate goal upon graduation — employment or acceptance into the military for further 
training in their occupational area. 

Ancillary services. As with the elementary and middle school projects, the projects that 
served high school students typically included a number of non-educational, or ancillary, 
services to support their participants' educational experiences. The range of these services 
tended to reflect the differing needs of high school students and dropouts. Six of the seven 
projects, for example, offered some form of counseling services. (The seventh, Memphis, did 
not include a formal counseling component per se, although participants received 
considerable informal counseling from their JTPA job developers.) Des Moines, Ethete, and 
Carbondale provided individual counseling to participants judged to need this service, and 
Aiken and Coleman operated group counseling services as well. In the Denver-Discovery 
program, counseling was literally available upon demand, and the project's counselor assisted 
students with all manner of concerns, including approaching other school staff to join after- 
school athletic teams. 

Two of the projects, Carbondale and Coleman, provided case management and referral 
services. Particularly in the case of Carbondale, many of the panicipants who had dropped 
out of school also needed help in making living arrangements, applying for public assistance 
or child care, and arranging transportation to the project or to work. Only Coleman included 
child care in the service configuration, and it was available on the school grounds and used as 
well to provide students with vocational training opportunities to become child-care workers. 
Aiken, Denver-Discovery, and Memphis included a number of incentive or reward-oriented 
activities (pizza parties, special T-shins, field trips, recognition parties with parents as special 
guests); Memphis, which placed significant emphasis on panicipants' self-images, included 
mentoring and other activities intended to improve motivation and self-esteem. 

g:< 
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Unlike the elementary and nuddle school projects, only four projects had any parent 
component at all, and only two of those (Des Moines and Denver-Discovery) incorporated 
what might be considered meaningful parent components. In these two projects, each of 
which emphasized the panicipation of students new to the high school setting (i.e., 9th- and 
lOth-graders), staff made visits to each participant's home and made periodic telephone 
contacts with parents to discuss participants' progress in school and on their jobs. (Staff of 
the Memphis project also reported frequent contacts with parents, particularly when they 
picked students up to take them to job interviews, but the project did not include a formal, 
organized parental involvement component.) Similarly, the Coleman project organized 
activities for parents (and, in one case, had a parent and her sons as students in some of the 
same classes), but project officials reported that parents seldom attended. 

Absence of the parental involvement component from high school projects is not 
unusual. By this age, many youth often are more involved with peers than with families, and 
in some instances, project staff reported that parents had given up on (or became burned out 
with) their adolescent children, acknowledging their lack of control over their attendance and 
performance in school and elsewhere. This problem was panicularly prevalent among the 
students and dropouts served by the Coleman and Carbondale projects. In many instances, 
these youth were no longer living at home, and the projects' counseling staffs spent 
considerable amounts of time helping students with basic needs (e.g., finding places to live). 

Structural Changes in the School 

Five of the seven high school projects implemented their programs in some form of 
alternative environment. Aiken, an alternative school, was located in a building that formerly 
housed a school for special education students. The Coleman alternative school was also 
housed in a separate facility, with students bused from other school districts. The Des Moines 
project operated as a school-within-a-school, a structure that provided panicipants with a 
smaller, presumably more nurturing environment. The Memphis project was a Saturday 
school. Participants attended their regular high schools during the week and participated in 
the project on Saturday in facilities on the campus of a local public university. Finally, the 
Carbondale project had limited space in a high school serving 9th- and lOth-graders, with 
separate entry and generally self-contained facilities. 

These sorts of alternative structures and environments have both advantages and 
disadvantages for students at risk. On the one hand, alternative schools offer the advantage of 
providing environments in which different approaches can be tried in attempting to reach 
youth; classes are typically smaller, and the fomial structure of regular school can be relaxed 
in order to provide environments that may be more productive for students. This type of 
advantage may extend to the school-within-a-school model as well. On the other hand, this 
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type of structure segregates students, and stigma is often a problem. Students in Aiken, for 
example, were stigmatized by their former classmates (at their home schools) in part because 
the school had been a special education facility, and classmates thought (or pretended to 
think) that it was still a school for "dummies." Similarly, students attending the Coleman 
school may also have suffered from this sort of outside-the-program stigma— e.g., thek 
home-school teachers commented that they were pleased that the school existed but would 
not want to teach there. (It is insightful to note, however, that at least one of the teachers at 
the Coleman alternative school had enrolled her child in the program.) The project director 
indicated that this bias extended to the sending communities as well; the students who were 
sent were sometimes perceived by staff at the alternative school as dangerous or violent. 

On the other hand, the Memphis Saturday school, while also a separate project, did not 
remove participants from their regular home-school environments and hence did not identify 
them to their teachers and peers as at risk, or somehow different and less worthwhile. 
Further, the fact that the project operated on college campuses gave it a sort of special status 
in that many programs for gifted and talented youth are held in these types of settings, in 
addition to affording students the opportunity to become somewhat familiar with being at 
college. 

Comprehensiveness and Coherence of Project Services 

Table 13 summarizes the major project service designs that were used at these seven 
evaluation sites. While all included an academic component (typically remediation), only one 
implemented a strong parent component. Three projects incorporated career preparation 
activities, and two projects offered paid-work opportunities. All of the high school projects 
offered counseling, and all either provided a variety of student support services or had 
implemented systems to link participants to appropriate community service agencies. These 
strategies reflected the status of many participants in terms of their needs for human services, 
including, for example, housing or income assistance. The project designs thus supported the 
notion that older youth arc likely to have multiple problems that may impinge on their 
academic progress, or even their persisi..nce in high school. 
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Table 13 

Summary of Project Service Designs for High School Projects 


Projects 


Des 


Memphis 


Coleman 


Aiken 


Etbcte 


Denver- 

Discovery 


















Y 
A 


Y 
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V 
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A* 
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X 


Pon^nt Tni/r^luAmAnt 
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Y 
A 










X 


G>unseling 


X 




X 


X 


X 


X 


Career Pienaration 


X 


X 


Y 

/V 








PaidWoilc 


X 




X 
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Indicates accelerated, or enriched, academic instruction 



Key: 



Academics — Services may include remediation in basic skills or emphasis on particular topics, as well as 
accelerated or enriched instructional opportunities 

Parent Involvement — Services may include parent training sessions, as well as activities aimed at ensuring 
parent knowledge and support for project efforts 

Counseling — Services may include individual or group counseling aimed at a wide-range of youth-related 
problems (e.g., problems with parents, stress) 

Career Preparation — Services may include career exploration, career awareness, or work-experience, as 
well as vocational training 

Paid Work — Services include provision of paid-work opportunities while students are enrolled in the 
project 

Attendance Monitoring — Services include direct monitoring of student attendance on a daily basis and 
may include active follow-up of students who are absent 

Support Services — Services may include child care or parent training classes for students, as well as other 
services that are intended to make it easier for students (or prospective students) to attend class 

Linkages — Services include referrals to other community agencies (e.g., for housing or income assistance) 



The Des Moines project offered the largest array of services to participants, including 
reduced student-teacher ratios, formalized parent involvement, and an incentive system that 
predicated paid-work experience on appropriate school attendance and academic progress. 
Given that participants were in their early years of high school, in combination with the 
extensiveness of the services available, the project might well have been expected to show 
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relatively short-term improvement outcomes. (In fact, these outcomes were documented fr< 
1989-90 through 1991-92, and the project, as shown in Chapter 5, consistently achieved a 
higher retention rate of enrolled students at risk.) 
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Chapter 3. Outcomes for Elementary School Students 

Virtually no students drop out of elementary school; the law does not allow it However, 
the seeds of behaviors that lead to dropping out may be planted in elementary school; hence 
one focus of the SDDAP was on early intervention. This prevention approach enables 
educators to help students avoid problems and behaviors that can lead to still more serious 
problems as the students grow older. For example, deficiencies in basic skills can gradually 
create serious learning deficits that eventually cause students to leave school in frustration. 
By remediating those skiUs and helping students stay on track, the theory goes, schools can 
keep students engaged in the learning process, provide them with success, and motivate them 
to stay in school until they graduate. 

Because prevention is a serious concern of education at the elementary level, it is not 
surprising that student-participants in the SDDAP projects at this level are most often 
identified and selected on the basis of teacher recommendation. Ninety-five percent of the 21 
K-8 SDDAP demonstrations, for example, relied on teacher recommendation to identify 
students at risk, and 86% relied on their recommendation to select students for the project. 
Since prevention is often perceived as a need to provide students with extra services, it is also 
not surprising that, among the 21 SDDAP grantees providing services at this level, pull-out 
programs (used by 50%) and summer school (used by 48%) were the most frequently used 
instructional settings. Of the 50 K-5 SDDAP grantees, by far the largest percentages were 
involved in providing basic skills or supplementary instruction and counseling services to 
students, suggesting that the nature of the extra help provided in these pull-out and summer 
school services was related closely to helping students adjust to the demands of schooling. 

Data for the elementary school students participating in the four projects involved in the 
in-depth portion of the evaluation were obtained from several sources. For students at 
Gushing, Hannibal, and Shreveport, attendance data and grades were gathered from school 
records. At these sites also, self-evaluations of school performance (i.e., getting to class on 
time, working on class projects, doing homework, reading books or magazines) and parent 
involvement in school (e.g., talking to teachers, checking on homework) were obtained 
through one-time surveys of students in grades 2-6. Students in Kindergarten and grade 1 
were considered too limited in school experience and reading ability to complete surveys 
meaningfully, so no survey data are available for the Los Angeles students and the youngest 
students at Shreveport and Hannibal. For K-1 students in Los Angeles, teacher ratings of 
student performance in selected subject areas, as well as in areas relating to classroom 
behavior and extent of parental support, were relied upon to describe the gains made by 
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students, since for tiiese students school records data on absences and grade averages must be 
regarded as poor measures of engagement in school. 

Descriptions of each of the elementary projects are presented below, followed by 
summaries of the outcomes for students they each achieved. As a general frame of reference 
for considering individual project accomplishments at this level, the overall mean percent 
days absent for both the treatment and control groups at the three elementary sites serving 
students above the Kindergarten level (i.e., all but Los Angeles)— combined over both the 
pre-treatment and during-treatment years and across all these sites — was about 3%, or 
roughly five and one-half days of absences per school year. The standard deviation of this 
estimate was close to 4%, indicating a skewed distribution; that is, while many students had 
few absences, some students recorded many absences. The overall mean GPA at baseline and 
during the follow-up period for the elementary students at the three study sites (including 
both treatment and comparison students) was 2.64 or just about a B-average, with a standard 
deviation of .58. 
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Project Advantage 
Gushing, Oklahoma 



Overview of Project Activities, 1989-90 

Project Advantage was an intervention especially designed for students at risk in rural 
school districts. Operated by the Child Service Demonstration Center, a state-funded 
technical assistance center, the project served K-12 students in the Cushing Public Schools. 
At the secondary levels, the project featured homework assistance centers, tutoring, a 
summer program, and activities to assist students who were pregnant, drug abusers, or 
dropouts. The main emphasis of the project was on providing basic skills instruction to at- 
risk students in grades K-5, and these services were the focus of the present evaluation. 

Services provided to Project Advantage students in grades K-5 included after-school, 
small-group instruction in reading and mathematics, academic instruction in summer school, 
and special counseling. The project placed special emphasis on in-service training for 
teachers. Most of this training aimed to help teachers utilize alternative instructional 
strategies, to enhance learning for students with different needs, and to enable students to 
stay on track with others in their classes. Student self-esteem was also emphasized through 
counseling services provided by project staff and classroom activities featuring parents, who 
received special training with scripted lessons. 

Project staff. Staff for the K-5 project component included regular elementary school 
teachers who provided after-school and summer instruction, aides who assisted those 
teachers, and a counselor who focused on helping adults work with students and also 
provided individual and group counseling. The project director was on the staff of the Child 
Service Demonstration Center. She directed all of the project activities and provided in- 
service training to staff. 

Participant selection. At the K-5 level, the project provided counseling to 79 students, 
after-school tutoring to 84 students, and summer tutoring to 56 students. Students were 
selected because of failure to master basic skills, poor attendance, and problems with school 
adjustment. For after-school tutoring, teacher referrals and achievement test scores were used 
to identify students; parents were then contacted to see if they wanted their children to 
participate. The majority of the students served by Project Advantage were white, while 
approximately 20% of the total students enrolled were either Black or Native American. 

Project services. The elementary-level students in the study sample received an average 
of 15 hours of service over 34 weeks, an average of 2.6 hours per week. This number varied 
somewhat among students, depending on whether they participated in the after-school 
program only, or whether they also received counseling services and summer instruction. 
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(The summer program lasted for five weeks.) Basic skills instruction was based on different 
strategies to meet varying learning styles, with special efforts to make it more interesting and 
motivating than the regular classroom work. "Hands-on" activities were emphasized in both 
the after-school and summer programs, with manipulative materials, some purchased and 
others developed by project staff, widely used. Computers were used for word processing 
and language development, and the whole language approach was used for language arts 
activities. Much of the instruction was one-to-one (student and teacher or student and aide), 
and buses were relied upon to provide transportation for students. 

Counseling services centered on Project Esteem, classroom activities provided by 38 
parent volunteers for about 40 minutes every other week in 2.nd- and 4th- grade classes and in 
most 3rd- grade classes. The counselor trained the parents and coordinated their activities. 
Lessons were scripted and emphasized group interaction, such as role-playing and promotion 
of students' self-esteem. Teachers were available to help if needed, and they filled out 
evaluations of each lesson. The counselor also coordinated a Positive Recognition Program at 
each school, with a "blue-ribbon" student selected for recognition on a special bulletin board 
each week. Counseling for individuals and groups was also provided; some home visits were 
made. 

In-service training for teachers included Early Prevention of School Failure, the Career 
Awareness Program (CAP), Diversiiled Educational Experiences Program, and Project 
FOCUS, all National Diffusion Network (NDN) programs, and emphasized developing 
strategies for helping students who were not achieving because of poor skills, lack of interest, 
or behavioral problems. Other project activities at the K-5 level included monitoring 
achievement and attendance, use of Project CAP activities, and a drug abuse prevention 
program. 

Project Advantage reported that the key parts of their K-5 activities were providing a 
system-wide program of teacher in-service training and developing and implementing a 
comprehensive set of intervention strategies for children at risk. The latter included the 
tutoring programs, accommodating individual learning styles, and involving parents. 

Notable Changes in Operations During 1990-91 

Only minor changes were made in the project during 1990-91. The after-school and 
student self-esteem components remained in place, and summer school activities were held; 
in-scrvicc training for teachers was once again a major thrust of the K-5 project component, 
with particular emphasis on the whole language process. 

Status in 1991-92 

A new SDDAP grant made it possible for the array of services offered by Project 
Advantage to be replicated in three other school districts. Emphasis at the K-5 level 

ERIC ^ ^ 



continued to be placed on whole language for reading instruction. In addition, the summer 
program began to emphasize outdoor education, and after-school tutoring was changed both 
in reference and emphasis to after-school enrichment. The services of the project counselor 
were continued, and family counselors were added. Critical to the success of the project, 
according to the project director, was the perceived value of such program features as the 
homework assistance centers and bus transportation for after-school tutoring. She reported 
that the project had particular impact within the district on the use of parents in the 
classrooms. During 1991-92, she reported that 40 parents from the community participated, 
with pairs coming to classrooms every other week for 1.5 hours per week to present skits 
designed to build students' self-esteem. Involvement in the program, which also was reported 
to have generated community support and to have provided students with role models, grew, 
and in 1992-93, three elementary schools added site coordinators to ensure that participation 
from parents and other volunteers continued. 

Project Outcomes 

The evaluation involved two matched samples of 100 students each selected from 
Project Advantage and from a school in a neighboring district (see Table 3 in Chapter 1 for a 
comparison of the demographic characteristics of these samples). An assessment of baseline 
(i.e., 1988-89) absence rates and grade averages between these samples revealed no 
differences; students in both groups were absent about 3.2% (or about 6 days) over the school 
year, and the grade averages for the treatment and comparison samples were 2.77 and 2.82, 
respectively, on a four-point scale. Records data used in these baseline assessments were 
obtained for 95% and 92% of treatment and control students, respectively, at the site. 

In 1989-90, implementation specialists working on the evaluation team determined that 
improved self-esteem and academic achievement were possible short-term outcomes that 
Project Advantage might achieve. Given the chief focus of the project, improved academic 
achievement was also listed as a possible future outcome, indicating bur sense that the 
project would help sustain academic improvements over time. 

Student self-esteem. Although measured only once during the course of the evaluation, 
at the end of the first treatment year in spring 1989-90 with completed forms received from 
three-fourths of the treatment and control samples, scores on a version of the Rosenberg Self- 
Esteem Scale (1965) developed for young children showed no differences between Project 
Advantage students and the comparison students. The composite scores for both groups were 
at the high end of the possible distribution of scores for the measure, although the lack of 
nauonal norms for this version of the Rosenberg scale make these data difficult to interpret. 
One of the main strengths of the project as reported by staff and observed during the 
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implementation study phase of this evaluation did seem to be in helping students change their 
attitudes about themselves, to feel successful and to like school. 

Student academic performance. On student self-evaluations of performance, a 
majority of Gushing treatment students reported that they were doing better than the previous 
year on all four performance items included in the student survey: getting to class on time 
(56%), work on class projects (55%), homework (64%), and reading habits (57%). On 
getting to class on time and work on class projects, the percentages of treatment students who 
reported these improvements were significantly higher than the percentages of the control 
students who reported these improvements. The response rates on the items for the two 
groups were slightly higher than 75%. 

The after-school program in Gushing provided students with opportunities to do their 
homework and receive assistance as needed from project staff, which probably helps explain 
why the project participants perceived that they had improved in this area. These students* 
perceived improvements in getting to class on time and work on class projects were 
somewhat more difficult to account for since the activities were provided during the after- 
school hours and did not include systems for monitoring students' attendance or work in their 
regular, day-time classes. However, the additional attention from teachers during the after- 
school activities, with attention to helping students stay current academically and feel more a 
part of the school environment, may have been successful in improving student's attitudes 
about regular school. 

Along with these self-evaluations, we collected the yearly grade averages for the 
treatment and control groups for both the 1989-90 and 1990-91 school years. Sufficient data 
were obtained to permit gain-score analyses based on 74% to 92% of the students in the 
groups for each year. By the end of 1989-90, neither the treatment nor the control students 
had recorded gains in their grade averages when compared to the baseline year. By the end of 
the 1990-91 school year. Project Advantage students were shown to have maintained their 
baseline averages (i.e., 2.78 average in 1990-91 compared to an average of 2.75 for the 
baseline year), while comparison students' averages declined by about one-quarter of a grade 
point (i.e., from 2.77 in the baseline year to 2.50 in 1990-91). These results, which closely 
approach a statistically significant difference in gain scores for the two groups, may be 
indicative of the effectiveness of Project Advantage in keeping grade averages up (i.e., in 
helping students to avoid earning lower grades at the next grade level), since there is an 
indication from the data collected in this and other studies that grade averages tend to decline 
and absences tend to increase for students at risk as they progress through the grades. It is 
also true, however, that these results fail to provide strong endorsement for the Advantage 
treatment as an effective means of increasing academic performance. 
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other findings. Data on attendance collected for both the 1989-90 and 1990-91 school 
years evidenced no differences between Project Advantage students and comparison students 
when ccmpared to data for the baseline year. These analyses were based on more than 80% 
of the cases in the two groups. However, as noted above, Project Advantage students were 
more likely than comparison students to report they had improved in getting to class on time 
when surveyed in the spring of 1990. Overall absence rates for the student samples in 
Gushing were low (i.e., around 3%), and slight decreases in absences were recorded during 
the treatment period for both groups (e.g.. Advantage students' rates of absence dropped 
from 3.3% in 1988-89 to 2.9% in 1989-90.) 

Survey data collected from about 75% of each of the student samples during the spring 
of 1989-90 revealed no differences between Project Advantage students and comparison 
students on any of the assessed dimensions of parental involvement. Although 66% of the 
treatment students reported tnat their parents had often checked on their homework since the 
beginning of the year, only small numbers of parents were reported as having often talked 
to teachers or visited classes, and relatively few also were reported to have often limited 
student's TV viewing. Given the academic emphasis of Project Advantage, it is somewhat 
surprising that the parents of Advantage students were not engaged more actively in these 
types of activities, although the lack of difference between the samples in the reported 
incidence of parents helping with homework may be explained by the fact that Advantage 
students were less likely to have left the after-school tutoring sessions with any homework 
left to do; that is, in perceiving that their children had completed their assignments with the 
help of teachers, parents may have assumed their involvement in checking over the work was 
not required. 

Factors affecting realization of outcomes. Project Advantage augmented school-day 
services through the addition of after-school and enrichment activities. In addition, linkages 
with other service providers were relied upon to diversify the resources available to be used 
with students and their parents. To familiarize staff with these services and to help ensure 
that students were assisted with their classroom problems and needs, a strong in-service 
training program was established. 
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Youth Enrichment Program (YEP) 
Shreveport, Louisiana 

Overview of Project Activities, 1989-90 

The Youth Enrichment Program (YEP) provided after-school language development 
and enrichment activities in five elementary schools in low-income, high-crime 
neighborhoods. It served students who were at risk of educational failure in grades 1-3 and 
featured two principal components. The first component emphasized language development 
activities and reading instruction provided by certified teachers, particularly through the use 
of Peabody Language Development kits. The second component offered enrichment 
activities in drama, fine arts, music, and dance and was directed by a lead teacher and 
presented by community volunteers, including parents. During enrichment periods, students 
might also obtain assistance in completing their school homework. 

Parents were required to volunteer time to the project, to prepare snacks occasionally, 
and to provide their children's transportation. Parents also served on the Parent Advisory 
"Board and received handbooks, newsletters, and telephone "Tips for Parents" on interacting 
with their children at home. 

Project staff. The language development half of the program was run by certified 
teachers. According to school principals and project staff, these teachers were selected 
because they were the strongest primary teachers in their respective schools. Lead teachers 
were paraprofessionals who were responsible for the enrichment portion of the program. In 
addition, each enrichment program had an aide funded through a Neighborhood Youth Corps 
Grant (JTPA). The program also employed a certified physical.education teacher and a 
language development coordinator, and made use of many community volunteers. The 
executive director was responsible for the overall program, budgeting issues, and staff 
training and evaluation. 

Participant selection. YEP worked closely with the Caddo Parish School Board to 
select the five host schools. During 1989-90, a total of 251 children participated for at least 
one semester and there was a waiting list of students wishing to be enrolled. All were black, 
and between 90% and 95% of the children received free or reduced-price lunches. Students 
were selected for participation by principals and counselors, with priority given to those who 
had been retained or were achieving below grade level in reading. All of the students met 
Chapter 1 eligibility criteria. 

Project services. As an every-day after-school program, YEP provided students with an 
average of 12.5 hours of service per week for 15 weeks. The language development portion 
of the program, which ran for about an hour each day, used the Peabody Language 
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Development Kits. These kits consist of detailed, scripted lessons designed to increase 
emphasis on stimulation of overall oral language skills in standard English. Peabody lessons 
provide a range of activities stressing such skills as reception through sight, hearing, and 
touch; expression through vocal and motor behavior, conceptualization through divergent, 
convergent, and associative thinking. Other activities are designed to increase competencies 
in classification, description, listening, following directions, naming, sequencing, vocabulary 
development, problem solving, and comprehension. During the enrichment portion of the 
afternoon, students were also given an opportunity to complete their homework assignments 
with the assistance of lead teachers and aides. 

Parental involvement activities were a central focus of YEP. A Parent Advisory Board, 
with at least two parents from each site, met throughout the year. All parents were 
encouraged to volunteer at the program sites, and they were required to come and pick up 
their children at the end of each day's session. As noted previously, YEP pubUshed a parent 
handbook, monthly parent newsletters, and created a computer-driven telephone system to 
provide ongoing information about project activities and ideas for interacting with their 
children at home. 

Notable Changes in 1990-91 

Operations were virtually unchanged in 1990-91. The project continued to operate at 
capacity and had a waiting list of students at risk who wished to enroll. 

Status in 1991-92 

The Youth Enrichment Program continued to operate during the 1991-92 school year, 
and the local Board of Education voted unanimously to provide funds to continue it in 
1992-93. Contributions from other sources were also obtained, and additional funds were 
being sought. YEP was reported to be offering services in two additional sites in 1992-93, 
for a total of seven sites, with the same basic program of academic work and enrichment. The 
project director attributed the success of the program in obtaining support to presentations 
tiiat had been made to local groups, stories about YEP in local media, testimonials from 
parents whose children were involved in YEP, and positive local evaluations. 

Project Outcomes 

The evaluation randomly selected 99 of the 149 participants in YEP in the fall of 
1989-90, along with a comparison group of 100 students in the 1st and 2nd grades at two 
of the five schools from which YEP students were drawn. Insofar as possible, comparison 
students were selected who were on the waiting lisl(s) for YEP, although YEP-eligible 
students who were not on these lists were also selected. A possible source of non- 
comparability between the samples was that parents of children who were eligible for 
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program participation but who were not on the waiting lists may have decided against 
program participation because they had no transportation to pick up their children at the end 
of the day. The number of these students included in the control group was small, but their 
lack of transportation may indicate important differences in socioeconomic status. 

Assessments of baseline differences on measures of attendance and grades during the 
1988-89 school year, which were based on data collected for just about two-thirds of the 
students sampled, revealed no differences between the groups. Percent days absent for both 
groups was about 2% during that year, and grade averages for both groups were about 2.7. 

Improved academic performance was projected following the 1989-90 implementa- 
tion study site visit as a plausible short-term outcome for YEP students. Specifically, 
improvements in completing homework were envisioned, as were improved grades and 
increased parental support. 

Student academic performance. First-graders in Shreveport were not asked to 
complete surveys, but more than half of the just under 50 treatment students in the 2nd grade 
reported improvements in four performance areas: getting to class on time (71%), work on 
class projects (63%), homework (70%), and reading books or magazines (54%). The high 
ratings for homework likely reflected the project's focus on helping students with their 
assignments. The numbers of treatment students reporting improvements in their work on 
class projects were significantly higher than the numbers of control students so reporting, 
although no special monitoring systems were in place at YEP sites that would check up on 
project participants' work in class at their regular schools and classrooms during the day. 
Project staff were, however, available to assist students with their regular-class assignments 
during some or all of the after-school periods. Perhaps this extra assistance, which was 
intended to prevent these students from falling behind their classmates, made it easier for 
them to take an active part in class activities. It is undeniable that keeping current with class 
assignments minimizes the chances of students tuning out class discussions or getting so far 
behind that they simply do not have the skills to tackle current assignments. The relatively 
lower percentage of students (54%) in Shreveport who responded that they were doing better 
in reading books or magazines is particularly interesting since the main thrust of this 
demonstration aimed to enhance participants' language development and reading skills. 

Comparisons of grade averages between the student samples based on records data 
collected for about two-thirds of each of the samples revealed no statistically significant 
differences between the YEP students and the control group. The 1989-90 minus 1988-89 
differences in averages were .13 for YEP students (indicating a .13 grade point increase) and 
.05 for the control students; the pooled standard deviation for the comparison was .58. The 
1990-91 minus 1988-89 differences in averages were -.04 for YEP students (indicating a .04 
grade point decrease) and -.24 for the control students (with a pooled standard deviation of 
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.58). The pattern of performance of YEP students mirrors that of the students in Project 
Advantage; that is, like Advantage, YEP students maintained their grade averages at about 
the same levels over the period of the evaluation, while the comparison students' averages 
decreased. Also like Advantage, while the performance of YEP students may not indicate 
that the intervention has the potential of increasing students' academic standing, the near- 
significance of the gain-score analysis for the samples does suggest that YEP may be having 
a positive effect. 

Parental support. As noted above, parents were required to spend some time working 
with the YEP project, and tips provided to parents by the project were intended to improve 
interactions with YEP students' in their homes. The student surveys used in the evaluation 
assessed parental involvement in five areas: (1) talks with teachers, (2) visits to classes, (3) 
help with homework, (4) checking on homework, and (5) limiting TV time. It should be 
clearly noted, however, that the teachers and classes referenced in the survey were not YEP 
teachers and classes; rather, the survey inquired about parental involvement with students' 
regular teachers and classes. For this reason, these two indicators would only be expected to 
change in the positive direction for YEP students if parental involvement in YEP encouraged 
extension of this behavior to the regular program, or if the tips provided by YEP were 
succ(;ssful in encouraging parents to become more involved in their children's regular 
educational program. 

On survey items related to parent involvement, almost half of the 2nd-grade treatment 
students reported that their parents often helped them with homework (49%) or checked on 
their homework (47%). This is particulariy understandable because parent volunteers helped 
students with their homework after school and were provided with special training on how to 
work with their children on school-related activities at home. Relatively few of the parents, 
however, v,ere reported as often visiting classes, and about as many were reported to have 
often talked with teachers as were reported to have often limited the amount of time for TV 
viewing. 

Our results in comparing YEP students to the control students (selected in the main from 
YEP waiting lists) suggest little or no effect of YEP activities on parental involvement in the 
regular school program. Parents of YEP students were reported to be no more likely to 
engage in talking with teachers, visiting classes, helping with homework, checking 
homework, or limiting TV time than were the parents of the control students. 

Other findings. In addition to school performance and parent involvement, we 
examined rates of regular school absences for YEP students and comparison students. No 
statistically significant differences in these rates were found for comparisons involving 
1989-90 minus 1988-89 rates or 1990-91 minus 1988-89 rates. Complete records data were 
available in making these comparisons for about two-thirds of each of the samples during the 
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treatment period (i.e., 1989-90 minus 1988-89). For the second follow-up assessment of gain 
(i.e., the assessment comparing 1990-91 to thp baseline), complete data were available for 
slightly fewer than one-half of each of the samples. In general, the rates of absences of either 
of these two groups over time varied by less than 1%. 

Absences for students in the YEP project decreased by almost one-half, from more than 
three days to less than two days, during the treatment period. Attributing any of this decrease 
in absences to YEP is not straightforward, however, as it operated as an after-school 
enrichment program focusing on the reading/language arts area. It is possible, certainly, that 
attendance at this project may have motivated students to attend scha:)l more regularly (e.g., 
so that they could participate in the after-school activities); however, the project had no 
specific policy requiring school attendance for project participation. 

Factors affecting realization of outcomes. Despite utilizing regular school teachers in 
its after-school programs, YEP remained very much an outside- school program. For there to 
have been increases in attendance, (reliably) higher grade averages, and increased parental 
activity with respect to the regular program, the effects of YEP would have had to "filter 
through" to that program. In such a case, strong links must be forged between the activities of 
the community-based organization (CBO) and the actual school performance of students. In 
the one area where YEP came closest to establishing these links, i.e., in assisting students 
with in-class assignments, students nearly (and relatively) outperformed their peers. 
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Eugene Field Elementary School 
Hannibal, Missouri 



Overview of Project Activities, 1989-90 

Eugene Field Elementary School decided to launch an accelerated school program in 
1988-89 as a result of strong administrative encouragement and a faculty vote. The program 
is based on setting high expectations for students, empowering and training teachers, and 
increasing the roles of parents and community members to create an environment that is 
challenging, supportive, and enjoyable for students. Decisionmaking is shared among all 
these stakeholders. At Field, separate committees, or cadres, addressed discipline and self- 
esteem, parent involvement, extended day activities, and curricular strategies. Each cadre 
included both teachers and one or two parent representatives; a representative from each 
cadre and a parent from the district sat on the school steering committee. The principal 
coordinated the program. 

The Field accelerated program emphasized whole-language instruction, the use of 
language skills across the curriculum, and the building of self-esteem. Extended-day 
activities were offered for one and a half hours after school, and students received tutoring 
services or attended enrichment classes during this period. The Field faculty received a 
considerable amount of in-service training to help them implement the accelerated school 
model. 

Project staff. The entire faculty was involved in furthering the school's accelerated 
program, though teachers were not required to participate in governance committees (i.e., 
cadres). The coordinator of the program, as noted above, was Field's principal. He was held 
accountable for student success by the central district office, though he and his staff were not 
deprived of primary decision-making authority. 

So that they might adequately dischai'ge their new set of responsibilities as teachers at 
an accelerated school, the faculty at Field received a substantial amount of training provided 
through the auspices of the state and with the assistance of persons intimately familiar with 
the accelerated schools concept. In addition to classroom management and self-esteem, 
common in-service topics included consensus-building, teaching in a multicultural context, 
and implementing the accelerated approach to curriculum and instruction. Most of the 
teachers attended two-day workshops four times a year; some received even more training. 
Indeed, ten members of the faculty attended a state-sponsored, five-day summer workshop 
given by representatives of the Accelerated Schools Project at Stanford University. 

Participant selection. Field enrolled 279 students in grades K-5 in 1989-90, all of 
whom were considered participants in the school's accelerated program. These students were 
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drawn from among the poorest and most heavily minority populations in Hannibal. The 
majority of enrolled students were white, with approximately 15% of total enroUees drawn 
ftx)m minority populations. 

Project services. According to the framers of the Accelerated Schools Project at 
Stanford University, it is essential for schools serving large numbers of students at risk to 
utilize all available assets and to support teachers and parents by increasing their 
responsibilities. At Field, teachers shared in school decision-making through participation in 
various committees. Separate committees were formed to address discipline/self-esteem, 
parental involvement, extended-day activities, and curricular strategies. Representatives of 
each committee as well as other teachers and the school principal sat on a school steering 
committee. The latter committee set the agenda for meetings of the school faculty, which 
voted on most issues affecting the school. 

On each of these governance committees, a spot was reserved for a parent 
representative. Parents were also actively encouraged to attend teacher-parent conferences 
and other school functions. Further, the faculty were active in soliciting support from local 
businesses and community organizations. These latter efforts yielded various benefits for the 
school, including an increase in the facilities available for extended-day activities. 

Finally, the one component of Field's program that most directly involved dropout 
prevention was the institution of the intervention team. Composed of two teachers, a 
counselor, a special services teacher, and the principal, this team met with the parents of any 
student who had been identified by his or her teacher as demonstrating significant academic 
or emotional difficulties. The group brainstonned strategies to be used to help the student. If 
these strategies, once implemented, did not appear to be effective, the child would typically 
be referred to specialists for evaluation. 

Because the program was virtually synonymous with the daily school experience, 
students received approximately 32.5 hours of services each week of the 36-week school 
year. 

Notable Changes in Operations During 1990-91 

Representatives of the Accelerated Schools Project have suggested that full 
implementation of an accelerated school model takes on average from six to seven years (see 
Levin, 1991). No major changes were made at Field during 1990-91, the second year of 
project implementation and the third year since work on planning the program and training of 
staff and parents had begun; faculty and parents continued to be actively involved in project- 
related activities. A nominal fee previously charged for the after-school program — $.50 per 
day per student — was reduced in 1990-91, with the result that program attendance more than 
doubled. 
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Status in 1991-92 

The accelerated school model continued to be implemented at Eugene Field School, but 
due to limited funding, training for teachers, administrators, and. parents was limited. Teacher 
planning time, which was a key component to assisting the work of the cadres (i.e, teams of 
parents and staff members focusing on such areas as student self-esteem), had to be reduced; 
15- to 30-minute meetings of these cadres were now being held before school. To ease the 
funding problem, grant applications were being written by the principal to obtain additional 
monies for support of various project components. 

During 1991-92, daily language classes of an extra hour and a half were added for 
Kindergarten students, either before or after school. Five cadres continued to function, and 
parent involvement reportedly had increased. Again according to local reports, the school 
was causing other schools in the district to re-examine some of their policies, and at Eugene 
Field, fewer students were reported to have been held back in 1991-92 than in previous 
years. In assessing overall program success for students, the principal referred to the 3rd 
grade cohort at Field (considered as his "thermometer group"); whereas 52% of this group 
was previously on grade level as measured by the California Achievement Test, that number 
rose in 1991-92 to 89%. 

Project Outcomes 

A sample of 100 students was selected randomly from among students enrolled in 
Eugene Field elementary school in the fall of 1989-90. The comparison group for the 
evaluation was drawn from an elementary school within the same district as Eugene Field 
that served approximately similar numbers of low-achievers who were receiving AFDC 
assistance in the form of reduced-price or free lunches. This comparison group was selected 
to match the grade-level distribution of the students in the demonstration sample, in addition 
to matching that group on the key demographic variables of gender, race/ethnicity, and age 
differential with respect to grade (see Chapter 1). 

Comparisons of absences and grade averages among the two samples during the 
1988-89 baseline year were hampered by the unavailability of these data for many students. 
Based on information that was collected for about half of each of the student samples at the 
site, comparison students were found to have recorded a greater number of absences — i.e., 
4.5% days absent (about 8 days) during 1988-89 as compared to 2.8% days for Field 
students. There was no difference between the grade averages for the groups for the same 
period. (When adjusted for nonresponse, neither absence rates nor grade averages for the 
samples during the baseline year were found to be significantly different.) 

The 1989-90 implementation study determined four possible short-term outcomes for 
this project: (1) improved academic performance, (2) change in school governance 
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structure. (3) improved staff capability, and (4) increased parent involvement. Two of 
these areas, which refer directly to student-related accomplishments, improved achievement 
and increased parent involvement, were the major focus of the evaluation. Data collection 
problems involving the student survey forms, however, made it impossible to assess the 
effects of project activities on increasing the involvement of parents. It is, however, an 
indication of parent support generally for the Field program that, as noted previously, parents 
remained involved in school-committee activities. In addition, reports available from School 
officials estimate that as many as 80% of Field parents were attending parent-teacher 
conferences at the beginning of each semester. 

Student academic performance. Based on data for about half of each of the student 
samples, comparisons of students' grade averages in 1989-90 and 1990-91 to those in 
1988-89 (the baseline year) show no differences between the academic performance of Field 
students and the comparison students selected from a neighboring school. The grade averages 
for both these groups remained just about the same over the two periods; beginning in 

1988- 89 with 2.4 grade averages, both groups showed slight gains — to 2.5 averages — in 

1989- 90 and no gains at all by the end of 1990-91 (when their respective averages were 2.35 
and 2.44). Of course, grade averages do not reflect directly any changes in the nature of the 
curricula offered at the two schools, such that a 2.4 grade average earned at a school with a 
more challenging, accelerated curriculum might indicate a substantially higher level of 
attainment than a 2.4 average (or possibly an even higher average) earned at a school with a 
less ambitious academic program. The evaluation did not formally assess differences in 
curriculum content between Field and the comparison site. 

Other findings. Comparisons of absence rates for Field students and those students at 
the comparison site also reveal no differences between the groups. From 1988-89 to 1989- 
90, Field students decreased their rates of absences by .1%. From 1989-90 to 1990-91, 
however. Field students increased their absence rates by .95%. Over the same periods, 
comparison students first decreased their rates by 1.5% (in 1989-90) then increased their 
rates by 2.9%. Evaluating 1990-91 minus 1988-89 differences in these rates statistically, 
controlling for race/ethnicity, gender, and age with respect to grade level, the mean 
differences for the groups are not significant at p<.05. 

Field did report impressive results in terms of student participation in extended-day 
activities. In particular, roughly 40% of the students were estimated to have attended the 
after-school tutoring sessions, and most students were reported to have attended some type 
of extended-day activity approximately two or three times each week. 

Factors affecting realization of outcomes. The most salient feature of this project with 
respect to student outcomes was that a reorganization of the school decision-making process 
was under way, which, while it did produce a few new programs in the short term, was 
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primarily aiming for substantial change over a seven-year period. In assessing performance 
changes over only two or three years, at the outset of the period, this evaluation really was 
not able to give the effort time to fully accompUsh its purposes. Such gains as may have beei 
evidenced at this site would seemingly have been attributable only to efforts associated with 
start-up of an accelerated school. 
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Kindergarten and Elementary Intervention Project (KEIP) 
Los Angeles, California 



Overview of Project Activities, 1989-90 

The Kindergarten and Elementary Intervention Project (KEIP) was developed by the 
School Mental Health Center of the Los Angeles Unified School District as a means of 
reducing behaviors in Kindergartners and Ist-graders that may eventually lead to dropping 
out of school (e.g., extreme aggressiveness). The project was staffed by a psychologist, 
licensed social workers, and on-site coordinators, who together recruited, trained, and 
supervised a corps of 250 unpaid volunteers who spent time with specific students in 
classrooms throughout the district. KEIP operated in 24 elementary schools that fed into the 
LAUSD middle and high schools with the highest dropout rates. Typically, teachers in two 
participating classes from each school referred three students per class to the project, based 
on the students' inappropriate school behaviors and other criteria listed on a referral form. 

Parent education and involvement was a primary focus of the project, with problem- 
solving classes for parents and individual and group counseling services available from the 
project's social workers. The social workers and on-site coordinators also provided parents 
with referrals to other agencies and services as needed. The needs of students and parents 
whose first language was not English were addressed to the extent possible: some parent 
classes were offered in Spanish; some bilingual volunteers were paired with limited English 
proficient students. 

Project staff. The project director was a full-fime psychologist in the School Mental 
Health Center, Kut devoted pan of her time to the development and management of KEIP 
Each of the two full-time licensed social workers, called project "interveners," had the 
responsibility for 12 schools within a geographic area of the school district. Each of the 24 
participating schools had an on-site coordinator who was a paraprofessiona) , trained by the 
director and the two project interveners (social workers). The paraprofessionals were usually 
suggested by the schools as active parents or community members who would be especially 
adept at working with the parents of students in the KEIP program. Teachers who 
participated by referring students and supervising volunteers were not actually project staff, 
although their cooperation and agreement with the goals of KEIP were deemed critical to the 
overall effort. 

The 250 volunteers, who were recruited from high schools, community colleges, 
undergraduate and graduate psychology and education classes, senior citizen centers, and 
other community agencies, made up the greatest numbers of staff. Their work was unpaid; 
however, some of the colleges they attended gave them credit for internships and other field 
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experience. All volunteers attended the orientation to the program and additional training 
sessions throughout the year, but much of their training came informally through discussions 
with the responsible teacher, on-site coordinator, and the project intervener. 

Participant selection. After being informed about the project in a meeting for primary 
teachers, classroom teachers who wanted to participate selected up to three students on the 
basis of observed behaviors. Students were referred on a form that asked teachers to identify 
any students in the class who evidenced need(s) for extra support and to note why each 
student was a concern— aggressive, extremely shy, underachiever, overactive, unnwtivated, 
or other reasons that might cause schooling to be an unhappy or unproductive situation for 
the student. From the list of those children whose names were compiled, each teacher then 
selected the three who were of greatest concern. Project staff then reviewed the nominations 
and selected those who appeared most in need. Teachers were free to identify others as the 
year progressed, but, for the most part, students were paired with the same volunteers for at 
least one semester. (Volunteers from the high schools and colleges, however, may have had 
to change their commitments to the youngsters when their own school schedules changed.) 
Approximately 75% of KEIP participants were minority students, with Hispanic students 
comprising the largest share (i.e., approximately 60% of total enrollees). 

Project services. The project had three major components: service to the parents of 
project students through problem-solving meetings conducted by on-site coordinators; 
service to the students that was supportive and non-disruptive because it was given in-class 
by caring volunteers; and specialized in-service training for teachers that was provided by the 
project interveners. The on-site coordinators themselves may have been parents of students in 
trouble at school, and by becoming active participants in helping their own children, it was 
intended that these parents learn the skills to help others. When appropriate for the general 
school population, bilingual coordinators were selected, and parent groups were conducted in 
both Spanish and English. (In some schools, as many as 30 different languages were spoken 
in the homes of student body members.) 

Compared to other SDDAP projects serving elementary school students, the intensity of 
the KEIP treatment was low. The average time that students received support from volunteers 
was about 10 hours per week for 30 weeks. Whether or not a kindergarten student continued 
in the program for the 1 st-grade year was determined by his or her Ist-grade teacher and the 
availability of community volunteers, although referral services remained available for all 
students. No follow-up services from community volunteers were typically available for 
students moving on to grade 2. Some of the volunteers, particularly those from the 
community, were bilingual and were paired with students with limited English proficiency. 
Volunteers were not tutors; rather, they gave students support, encouragement, and 
explanations, and they enhanced listening and communication skills under the supervision of 
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the classroom teachers. They learned to be helpers with individual children, who needed to 
strengthen their ability to cope with the academic and interpersonal/social demands of 
school. 

The two licensed social workers assisted the teachers in identifying students and met 
with parents individually to work on specific major problems or in groups to discuss 
solutions to common problems. These "interveners" also trained and helped the on-site 
coordinators, in planning and conducting parent groups and in coping with especially 
difficult student problems. Together, the interveners and the on-site coordinators also 
arranged for other services to be provided, such as referring parents to other agencies, school 
psychologists, case workers, or the LAUSD Mental Health Division. 

Notable Changes in 1990-91 

No significant changes were made to the program in 1990-9 1 . Volunteers from a wide 
range of community groups continued to participate; attendance at parent groups was good; 
social workers and other staff provided assistance and referrals for both students and parents. 
Status in 1991-92 

The Kinderganen and Elementary Intervention Project was expanded through a new 
SDDAP grant that reponedly had strengthened the parent component and increased the 
project's focus on the home environments of students. The program continued to recruit and 
train community volunteers to work with three high-risk students per class; however, services 
were reported to be less static, as students were permitted to move in and out ~f the program 
more frequently. The program also was reponed to have had an impact in the financially 
pressed district. For example, some program personnel were retained with district funds and 
others rehired despite severe funding cuts; a School Board member, using funds previously 
earmarked for hiring an assistant, arranged to fund a KEIP school in her area instead. 

Project Outcomes 

The evaluation included 1 14 students selected in 1989-90 from 14 of the 24 
demonstration schools that operated on a traditional nine-month school calendar. In all, KEIP 
participants fi-om 28 classrooms were involved in the evaluation. For the control group of 
Kinderganen students, 41 of the 82 students already identified by local evaluators as 
appropriate comparisons were selected. In addition, 42 Ist-grade students were identified by 
KEIP staff and classroom teachers for the comparison group. Insofar as possible, all 
members of the 83-student comparison group were perceived by teachers as exhibiting 
personal and social problems comparable to those of the demonstration students. 

The single evaluation instrument used was comprised of five-point rating scales on 
seven dimensions: citizenship, peer relations, class panicipation, language development, 
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mathematics development, parent cooperation, and parent involvement. These scales were 
completed by classroom teachers for each of the program and comparison students taking 
part in the evaluation. In 1989-90, teachers completed their ratings in the spring and were 
asked to provide two separate ratings on each dimension for each student at that time: one 
describing the performance of the student (and his or her parents) at the beginning of the 
year, and one describing his or her performance (and parent performance) at the end of the 
year. In the spring of 1990-91, the current teachers of each of the program and comparison 
students were asked to complete one set of ratings for each child, describing his or her 
behavior (and his or her parents' behavior) as of the end of the 1990-91 school year. 
Response rates in general were acceptable, with ratings collected for more than 75% of the 
student samples in 1989-90 and for 50% of the student samples in 1990-91. 

Using the first set of ratings as a measure of baseline differences (i.e., the ratings 
provided for the beginning of the 1989-90 school year), we found no statistically significant 
differences between the KEIP students and the comparison students on any of the seven 
dimensions. In short, based on the judgments of their teachers, program and comparison 
students in Los Angeles differed little in citizenship, relations with peers, classroom 
participation, language development, math development, cooperation of their parents, and 
involvement of their parents in school activities. 

The baseline, first-follow-up (i.e., during treatment), and second-follow-up 
(i.e., 1990-91) teacher ratings for the students served by the Los Angeles project arc 
presented in Appendix A. In reviewing these tables, note that lower ratings indicate teacher 
judgments of higher performance levels. 

The short-term outcomes that were judged io be plausible by the 1989-90 
implementation study team included increased parental support and improved self-esteem 
and academic performance among students. Possible future outcomes included improved 
social and academic adjustments to school. As no measure of students' self-esteem was used 
by the evaluation for K-1 students, assessments of whether gains were actually achieved 
were confined to the areas of parental support and improved academic performance. 

Parental support. Comparisons of gain-scores based on teachers' ratings of parent 
cooperation and parent involvement evidenced no differences between KEIP students and 
comparison students. This is true for both 1989-90 minus 1988-89 and for 1990-91 minus 
1988-89. In the opinions of their children's teachers, the parents of KEIP students had 
become no more cooperative or involved in school activities over the study period than had 
the parents of the students who were selected for the control group. However, our 
observations of the parent-training classes conducted by KEIP did evidence considerable 
enthusiasm among parents for this activity. In unstructured interviews conducted with parents 
at these classes, parents also expressed their appreciation for the efforts of the KEIP social 
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worker. Approximately 70-80 parent group meetings were reportedly held each year, with a 
total of about 1,000 parents attending (30-40% of die total number of parents possible). 
Some parents of former participants (currently 3rd- or 4th-graders) were reported to have 
continued attending parent groups, although their own children were no longer in the 
program. 

Student academic performance. Three dimensions on which teachers were asked to 
evaluate students related directly to student academic performance: class participation, 
language development, and mathematics development. Considering the gain-scores on these 
dimensions, KEIP students were judged to have outperformed their peers on one dimension, 
language development, when baseline ratings were subtracted from end-of-the-year 1989-90 
ratings. When baseline ratings were subtracted from 1990-91 ratings, however, there was no 
indication that this perceived achievement in language development persisted, and no greater 
achievement by KEIP students on either of the other two dimensions became apparent. 

Other findings. Gain-scores based on teacher ratings for 1989-90 suggested that KEIP 
students had shown greater improvement since the baseline measurement in peer relations 
than had comparison students. Like the finding related to language development, however, 
this finding related to peer relations also did not persist into 1990-91. On the remaining 
measure, classroom citizenship, no differences were found between the teacher ratings for 
KEDP students and the comparison students. 

Factors affecting realization of outcomes. The KEIP project involved coordination of 
different units within the LAUSD, and assistance in the form of in-service training in 
coordination-related topics was provided to teachers who were involved with the effort. 
Perhaps as a result, teachers were able to work constructively with community volunteers in 
the classroom and noted student gains for students in selected areas.. By 1990-91, most 
teachers no longer had community volunteers available to assist with selected students. 
Similarly, while some parents may have continued attending project-sponsored training 
sessions, the majority ceased to attend. In short, while the treatment was active for students, 
they responded; however, the can7-over effect was not discernible. 
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Summary and Interpretation of Findings for 
Elementary School Projects 



Summary of Findings 

Table 14 summaxizes the principal findings from the evaluation for the four elementary 
school projects operating in Gushing, Shreveport, Hannibal, and Los Angeles. 
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Table 14 

Summary of Improvements for Students Served in 
the Four SDDAP Elementary School Demonstration Projects: 
Measures Based on Teacher Ratings, School Records, and Student Self Reports^ 


Gushing, OK 
1989-90 Student Cohort 
First-Year FoUow-Up 

• Better at getting to class on lime 
compared to previous year (ss) 

• Better at working on class projects 
1 compared to previous year (ss) 

8 Second- Year Follow-Up^ 

1 'No improvements found 


Shreveport, LA 
1989-90 Student Cohort 
First-Year FoUow-Up 

• Better at working on class projects 
compared to previous year fss) 

Second- Year Follow-Up^ 

• No improvements found 


B Hannibal, MO 

1 1989^90 Student Cohort 

n First-Year FoUow-Up^ 

1 'No improvements found 

R Second-Year Follow-Up^ 

1 'No improvements found 


Los Angeles, CA 
1989-90 Student Cohort^ 
First- Year FoUow-Up 

• Greater language development (tr) 

• Improved relations with peers (tr) 
Second- Year FoUow-Up 

• No improvements foL'nd 



1 All estimates of improvements for students in the demonstraUon projects are relati ve to comparison 
students; that is, means or rates assessed at one point in lime or gain-scores for students in the 
demonstraUons are determined to be improvements by comparing them to similar measures for 
comparison students. Aside from information on absences and grades, which were collected from school 
records (sr) at all the sites, all other measures are derived either from teacher ratings - tr (at the Los 
Angeles site) or student surveys - ss (at the Gushing, Shreveport, and Hannibal sites). Survey dau for the 
1989-90 student cohorts are based only on one administration of the survey insirument"i.e., during the 
First- Year FoUow-Up. Complete information for all findings is provided in Appendix A 

2 Only possible improvements in absences, grades, and dropout rales were assessed for this follow-up, as 
student survey data were not available for analysis 
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Only teacher ratings were gathered to assess student performance at this site, as students were in grades K- 

In general, few significant differences on only a limited selection of indicators distinguish the 
school performance (or self-reports) of students receiving services from these SDDAP 
demonstrations and those not receiving services. 

All four of the projects emphasized the goal of improving students' academic 
performance either through skills development or enrichment activities. In Ciishing, the data 
on grade averages for Project Advantage suggest that by 1990-91, program students had 
succeeded in maintaining their baseline-year averages while the comparison students lost 
ground. At the end of the 1989-90 school year. Project Advantage students also more 
frequently reported than did comparison students that they were doing better in working on 
class projects (and on getting to class on time) as compared to the previous year. The efforts 
of the YEP project in Shreveport to offer after-school assistance on in-class assignments and, 
more generally, on skills development related to classroom work, produced siinilar results 
during the evaluation period. The data on student grade averages, for example, suggested that 
YEP students were more able than comparison students to maintain their baseline averages 
over a two-year period. While YEP participants' averages remained at about 2.70, 
comparison students' averages dropped by about one-quarter point. YEP students also more 
frequently reported on the 1989-90 survey that they were doing better (than in the previous 
year) on class projects. 

In Los Angeles, social behaviors as well as academic skills were stressed, and teacher 
ratings during 1989-90 indicated that progress had been made in both areas. KEIP students 
were judged to have improved more than the comparison students in their relations with 
peers and in language development. By 1990-91, however, the extent of accomplishment in 
these areas was not statistically significantly different from that observed in the comparison 
students. 

Clearly, much of the rationale for dropout prevention programs at the elementary grades 
is based on helping students avoid eariy failures that may lead to increasing disengagement 
from school in later years. The evaluation data for Gushing may be noteworthy in this respect 
because they suggest that the Project Advantage combination of one-to-one instruction using 
manipulative materials (including computers) together with special activities to motivate, 
interest, and praise student efforts may be successful in helping students to sustain 
achievement behaviors over the longer term. The results from the other three demonstrations 
provide less evidence of the effectiveness over time of the particular strategies that were 
used. 



ERIC 



88 



Ci 



Interpretation of Findings 

The overarching questions posed by the federal mandate for this evaluation concerned 
the effectiveness of various organizational structures and program strategies for dropout 
prevention. These four elementary-level projects provide several insights on what may be 
important preliminary answers to these questions. 

Organizational structure. What the experiences of the Hannibal project may suggest is 
that school restructuring that involves a re-organization of decisionmaking and governance 
systems takes time and may delay the achievement by students of new levels of school 
performance. As noted six or seven years is the expected period for full/successful 
implementation of an accelerated learning approach school-wide. After three years (i.e., one 
planning year and two years of implementation), Eugene Field students may not have been 
exposed to a fully accelerated curriculum that was offered in a consistent manner across the 
grades and widely supponed by teachers and parents. 

Similariy, although the involvement of community-based organizations in promoting 
students' achievement may be a key strategy for effectively targeting scarce resources on 
educational priorities, programs such as Shrevepon's Youth Enrichment Program may need 
an even more integral relation with the schools to affect classroom performance, and building 
this sort of relationship may take additional start-up time. Both Project Advantage and YEP 
offered strong after-school components that aimed to assist students in ways that would 
benefit them during the school day. In contrast to YEP, however, Advintage was an 
extension of the regular school program, with proj;;ct planning and scheiluling of activities 
carried out in close coordination with school personnel. Although it offered a distinctive 
program. Advantage was very much part of the public school system. As a consequence, this 
sort of program may have fewer start-up problems and fewer difficulties generally in 
working with existing school-based programs. 

The two early indications of positive results from Los Angeles' KEIP intervention for 
K-1 students seem to have resulted from an apparently successful effon at coordinating 
services between distinct units within the Los Angeles Unified School District. In bringing 
together mental health specialists, trained volunteers, and teachers, KEIP's early success may 
have been due in pan to the fact it provided training and support for educational staff that 
clarified just how the program would benefit classroom goals and activities. In addition, 
KEIP respected teachers' judgments and abilities to help students. For example, teachers 
were involved in the identification of children to receive services, and they were consulted in 
the assignment and scheduling of volunteer visits to the classroom. The KEIP approach 
suggests a somewhat higher level of interdepartmental relations, one much closer to service 
integration. 




To sum up, organizational restructuring, including restructuring for interagency activity, 
takes time and may reduce the short-term effectiveness of programs in providing direct 
services to students. Over a limited time period, e.g., one, two, or three years, it may be likely 
that projects attempting restructured service approaches will not show significant gains in 
student achievement. In contrast, projects or programs that add-on specialized offerings to 
existing service configurations may be more likely to evidence successes in die shon-term. 
These efforts are less likely to become entangled in administrative or bureaucratic 
complexities and are more likely to begin serving students immediately upon receipt of 
funding. In the context of education, the most straightforward "add-ons" are likely to be 
designed for classroom programs. Although they may be somewhat more complicated to get 
started, these "add-ons" may also work effectively when administered by outside-school 
agencies. The key in these cases to early success may be training and support for school- 
based practitioners with whom the community-based program sponsors must work. Of 
course, none of tiiis is to say that restructuring and interagency efforts are not the more 
effective approach over the longer term. To assess their effectiveness, however, a longer 
evaluation period is required. 

The elementary projects that provide some indication of a positive result for student- 
participants also are noteworthy in that they feature organizational structures that minimize 
possible labeling or other sources of stigma for students. Project Advantage, for example, 
focused its energies during after-school and summer sessions. Similarly, YEP combined an 
after-school care arrangement with learning enrichment and school-help activities. Finally, 
KEIP brought volunteers4o the classroom to befriend all students (spending more time by 
design with particular students in need) while it worked outside of the classroom with 
students' parents and teachers. These sorts of structures provide help seamlessly to students, 
in ways that seem both natural and helpful. Most importantly, stuucnt-participants are not 
made to feel less able or more subject to ridicule from classmates as a result of being 
involved in the programs. 

Effective strategies. Table 1 1, presented in Chapter 2, reviewed the service designs at 
the four elementary sites involved in the in-depth evaluation. At the rather general level of 
that table, and given the rather scant evidence of consistent success provided in this chapter, 
it is important to note that the two projects offering the greatest variety of service elements, 
Gushing and Los Angeles, were also projects whose students ev .denced some gains from the 
demonstrations, at least in terms of the measures used by this evaluation. (In the later 
chapters of this report, we will also see that comprehensiveness of services seems related to 
student outcomes more generally). Somewhat more specifically, as shown on Table 1 1. the 
Gushing and Los Angeles projects were the only ones to include counseling components as 
well as interagency links to a broader range of services for students. For younger students in 
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particular, these types of services, which may be brought to bear on a wide range of personal 
and health-related problems, may be key to achieving progress in learning-related areas. 

The involvement of parents in the Gushing, Shreveport, and Los Angeles projects must 
also be highlighted as a possible factor in helping to positively influence outcomes for 
students. In each case, parents and parent volunteers on these projects were provided with 
descriptions of their roles (i.e., what was expected of them) and trained or provided with help 
in carrying them out. In short, these projects assisted parents in understanding what parent 
involvement was to mean. Perhaps as a result, the projects reported high levels of parent 
participation. While evaluation results in this area are rather muted, it must be realized that 
student surveys — particularly surveys of very young children — are not among the most 
sensitive measures. In this case, the sustained participation by parents in the projects may be 
the most valid indication of success. Since sustained parental participation appeared also to 
be a feature of the Hannibal project, the communication of role descriptions to parents may 
not need to be any more specific than assignments to school-related communities that are 
invested with decision-making authority and encouraged to be active. 
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Chapter 4. Outcomes for Middle School Students 



The transition to middle school is stressful for many students: they go from the secure 
environment of the elementary school, where their day was generally centered in one room, 
with one teacher and the same group of students, to a larger building, where they go to a 
series of rooms, often a considerable distance apart, with teachers and classmates changing 
five or six times a day. Many students begin to turn off in middle school, and their behavior 
may reflect their increasing disengagement. For example, we found that rates of absence for 
middle school students on average are about 2% higher than for elementary school students. 
To increase students' commitment to their education, educators can design projects that 
attempt to address a wide range of student needs, including academic, psychological, and 
social needs. 

The manner in which these needs are addressed at the middle school level may be more 
similar to the approaches taken by elementary programs than high school ones. For example, 
at both the elementary and middle school levels the majority of SDDAP grantees reported 
usin^ more limited, or focused, service approaches, choosing to concentrate only on 
academics or on academics and personal counseling rather than a comprehensive mix 
including academic, counseling, social, and career-related services. Only 20 of 50 K-5 
projects and 28 of 64 6-8 projects reported offering comprehensive services in 1990-91. 
Projects serving grades 9-12 and re-entry students clearly preferred a comprehensive service 
strategy; at these levels, 37 of 61 projects and 23 of 42 projects, respectively, favored a 
comprehensive services approach. At the same time, however, the focused- service SDDAP 
projects operating at the grade 6-8 level were more likely to repon themselves operating at 
maximum capacity than were those projects offering more comprehensive services at this 
level (e.g., 59% versus 46% of the projects). This result was contrary to the findings among 
SDDAP projects at each of the other levels, where comprehensive service projects typically 
operated at higher capacities than did projects offering limited services only. Perhaps in 
"losing" the single-teacher single-classroom closeness of the elementary school and in not 
yet being ready for the faster-paced settings of the high school, middle school students are 
most easily accommodated by programs that work on specific learning-related problems. 

Five middle-school projects participated in the in-depth evaluation: Portland, Oregon, 
Broward County, Florida, Denver-Lake, Colorado, San Antonio, Texas, and the Bronx, New 
York. Each of these sites operated within the typical school environment and used a range of 
mechanisms to enhance students' learning experiences and help encourage their affiliation 
with school. In Portland, for example, counselors provided services to groups and, 
informally, to students in the many venues schools afford for one-on-one meetings. In 
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Broward County and San Antonio, pull-out programs were featured, with the Broward 
County activity niaking use of peer tutors. The Denver-Lake site utilized teacher-teams and 
block scheduling, while the Bronx demonstrations (there were two treatment settings) added 
instructional staff and provided after-school counseling and academic support. 

For the Portland and Broward County demonstration projects, outcome data were 
collected from two different student cohorts, i.e., two different pairs of treatment and control 
group samples: the first nohort, which was selected in 1989-90 and for which 1988-89 (or, in 
the case of Denver-Lake, 1987-88) was the baseline year for over-time comparisons, and the 
second cohort of students, which was selected in 1990-91 with 1989-90 as the baseline year. 
In the project descriptions to follow, we refer to these as the 1989-90 and 1990-91 student 
cohorts, respectively. 

Finally, as was noted in Chapter 1, the collection of outcome data from die Bronx, New 
York site was unsuccessful. Problems with the use of active permission forms, reluctant 
teachers, and difficulties in locating and accessing student records led to the termination of 
evaluation activity at the site by the end of 1989-90. In this chapter, we describe the design 
of demonstration activities at the site, but do not present information related to student 
outcomes. In contrast, for the four sites where evaluation activities continued through 1991- 
92, a variety of indicators were assessed and are presented in this chapter to describe the 
status and perceptions of the students participating in the demonstrations. Data were gathered 
on percent days absent, yearly grade averages, numbers of suspensions from school, dropout 
rates, self-evaluations of performance, student reports of parent involvement in school 
activities, and students' perceptions of the climates for learning at their schools; that is, the 
attitudes and behaviors of the teachers toward students, the types of disciplinary policies 
being followed, and whether students felt safe on their campuses and free from excessive 
disruptions caused by other students. 
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The Cluster Plan for Dropout Prevention 
Portland, Oregon 

Overview of Project Activities, 1989-90 

The Cluster Plan for Dropout Prevention was implemented through six Portland Public 
Schools programs serving K-12 students. The overriding intent of the plan was to empower 
students: to promote their success and retention in school by removing barriers — whether 
institutional, internal, social, or family-related— and providing the necessary support, skills, 
and enabling partnerships. The evaluation focused on the Cluster Plan efforts at George 
Middle School (CMS), which served students in grades 6-8. 

At this school, an intervention team consisting of the principal, a counselor, the vice 
principal who handled discipline, student services specialist, drug and alcohol specialist, 
school psychologist, and Children's Services Division (CSD) caseworker worked to identify 
students needing services and to develop appropriate service plans. Referrals for service were 
made both formally and informally, and a variety of individual and group counseling sessions 
and related activities (e.g., rehearsing and performing skits on drug or alcohol abuse) were 
carried out with students, who were drawn from a low-income neighborhood in North 
Portland with high rates of chronic unemployment and alcohol and drug abuse. In addition to 
group sessions and informal counseling and discussions with students, project staff often 
acted as student advocates, helping teachers to understand why students might have been 
having difficulties in their classes and assisting with the development of plans to resolve 
these difficulties. 

Project staff. At the district level the project was managed by the assistant director of 
student services. Cluster Plan coordinators met with her on a weekly basis. At CMS, the 
student services specialist reported to the principal and directed the efforts of the other 
Cluster Plan staff at the site. She supervised the student services center and arranged case 
assignments to staff in all referrals for support services. In addition, she provided direct crisis 
intervention, group support, and individual counseling services to students and their families. 

Because the project's goal district-wide was to promote success and retention in school, 
it also funded three full-time drug and alcohol counselors (one of whom was assigned to 
CMS) and a half-time attendance monitor. The Children's Services Division of the State 
Department of Human Resources assigned a caseworker to CMS to serve families whose 
children attend the school. Project Return staff, who were supported by SDDAP funds to 
intervene with students who were close to diopping out of school, worked closely with other 
staff at CMS; they were housed at a high school where a transitional classroom was provided 
for middle school and high school students. 
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Participant selection. The CMS attendance area was in an isolated, low-income 
neighborhood in North Portland. Reportedly, there was a high rate of chronic unemployment; 
the incidence of single-parent families or parents who were dysfunctional due to alcoholism 
and/or drug abuse was also reported to be high. The majority of the students receiving ' 
services were white (i.e., 70% of the total students served), while Native American students 
comprised the next largest group (i.e., approximately 13% of the students served). 

The Cluster Plan served all students who attended GMS, with some receiving more 
services than others. The student services staff participated in elementary school orientation 
for entering 6th graders, conducting mini-support groups, directing art activities, and 
discussing the range of services that were available. Every effort was made to help all 
students see the student services specialist and other staff as some of the resources that would 
be available to all students when they came to GMS. Once at GMS, students were referred 
for services both formally and informally. 

Project services. Counseling services to individuals were provided on an as-needed 
basis, and students who received intensive concentrations of services might have done so in 
two or three sessions per week for up to 21 weeks. Weekly meetings of groups to discuss 
alcohol and drug use were also called by intervention team members. In addition, counselors 
were available to talk to students informally between classes and on the school grounds 
during the lunch break and after school. Any of the students who had received services 
during the 1989-90 school year were eligible to continue working with project staff in 
1990-91, and many did. 

Notable Changes in Operations During 1990-91 

A key member of the George Middle School intervention team, the drug and alcohol 
specialist, was reassigned at the beginning of the 1990-91 school year. She was replaced by a 
new specialist with somewhat less experience working directly with middle school students. 
In addition, a new CSD caseworker was assigned. During our site visit to the school in 
1990-91, there was less obvious emphasis on an intervention team; instead, identification of 
students to be served and the provision of services to the students appeared largely to be the 
decisions of the specialist and the caseworker. In our ratings of the quality of project 
services, these staff persons in our judgment seemed to provide exceptional attentiveness to 
students' personal/psychological problems, their reactions to unexpected stresses in their 
personal lives, and the development of their self-esteem. A feature of the Portland 
demonstration that had been emphasized in 1989-90 — attendance monitoring — seemed to 
have been reduced considerably as a formal emphasis of the project in 1990-91. 
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status in 1991-92 

Oregon's Proposition 5 made district funding for dropout prevention activities 
extremely difficult in 1991-92 (the year following the SDDAP grant period), according to 
the district project director, but any parts of the Cluster Plan that could be fit into existing job 
descriptions were reported to have been continued. At George Middle School, counseling to 
combat drug and alcohol abuse was still offered, and Peer Helpers, a program to enable 
middle school students to assist peers and younger students, continued with existing staff. 
Liaison with the Child Services Division was also maintained, but on fewer days and with 
reduced hours. The district project director believed that the SDDAP project provided a 
reference point for the types of services students need, and she continued to seek funds to 
reinstate services. 

Project Outcomes 

The evaluation was focused on four samples of about 100 students each— two samples 
drawn in 1989-90 (97 and 99 students, respectively) and two drawn in 1990-91 (100 and 103 
students, respectively). The two samples of treatment students attending George Middle 
School were selected from among those 6th-graders who were identified for services at the 
beginning of these two school years. The two samples of comparison students were identified 
from among pools of comparable students (in terms of gender, race/ethnicity, and age 
■ differentials with respect to grade) in the 1989-90 and 1990-91 6th grades at Binnsmead 
Middle School, which is of comparable size and has comparable daily attendance patterns 
and numbers of students from lower socioeconomic backgrounds. 

Analyses of baseline-year differences between the treatment and control samples 
included in the 1989-90 student cohort indicated that the Binnsmead students (i.e., the 
comparison students) had recorded fewer absences during the 1988-89 baseline year, but that 
the grade averages for both groups of students were the same. The sample sizes available for 
these analyses (i.e., the numbers of students from the 1989-90 cohort for whom these data 
were available) were roughly 90 students per group. The same comparisons involving the 
treatment and control samples included in the 1990-91 student cohort evidenced somewhat 
similar results; students in the treatment group had recorded higher rates of absences during 
the 1989-90 baseline year (5.5% as compared to 3.1%), while grade averages for the two 
groups were not different. In addition, based on a survey of students in the 1990-91 cohort at 
the outset of the 1990-91 treatment year, GMS students served by the SDDAP demonstration 
reported reading fewer books or magazines, working more hours per week for pay, being less 
likely to have talked to their parents about courses or programs at school, and their parents 
were less likely to have attended a school meeting since the beginning of the school year. 
These r-tudents did, however, more frequently report that they had talked with a counselor at 
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the school regarding their school program. The baseline-year analysis of the 1990-91 student 
samples at the site were run using data for at least 90% of the students in each group. 

During the 1989-90 and 1990-91 school years, several areas were identified in which 
this demonstration project might plausibly realize achievements for students. These 
achievements included the more immediate improvements in students' self-esteem and 
attendance, as well as the longer-term improvements in students' academic performance 
and persistence to graduation. 

In large part, these determinations of plausible achievements were based on 
observational data that indicated the project had succeeded in increasing the availability of 
counseling services for students. These observations have since been confirmed by student 
survey data comparing the 1990-91 treatment and control group samples in Portland. 
Specifically, treatment students reported greater numbers of interactions with counselors for 
information or help with these matters since the beginning of the school year than did' the 
comparison students. 

In contrast, the survey provided no indication that students participating in this 
demonstration perceived any more frequent interactions during the year with school staff 
regarding counseling on personal problems, a finding that seemed contrary to our 
observations. One possible explanation of this finding is that the personal counseling was 
typically provided in very informal settings (e.g., on the school ground following the lunch 
break), and students may have been less likely to characterize their discussions with project 
staff during these sessions as counseling. 

Student self-esteem. Although for the 1989-90 student cohort the survey form for the 
evaluation was administered only at the end of the treatment period (i.e., in^Spring 1990), the 
data from the 10-item Rosenberg Self-Esteem Scale included in the survey evidenced no 
differences in the self-esteem of the George and Binnsmead students in Portland. Student 
survey data that were collected twice in 1990-91 from the 1990-91 cohort and permitted 
gain-score comparisons on the Rosenberg Scale also showed no greater gains in self-esteem 
on the part of demonstration project participants than were reported by the comparison 
students at Binnsmead. Similar results vere found when a second follow-up of the 1990-91 
student cohort was carried out at the end of the 1991-92 school year (i.e., when scores on the 
Rosenberg for 1991-92 were compared *.o 1989-90 baseline scores). 

Our observations and subsequent ratings of the implementation of this demonstration 
did suggest that less attention was paid by staff to self-esteem development per se than to 
counseling. For example, although students participating in the demonstration were 
encouraged to work together in putting on skits for elementary schools concerning alcohol 
abuse (the type of activity that would be more likely to win them recognition and praise from 
others), no direct efforts were made to increase students' self-esteem through such activities 
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as use of motivational speakers, specially designed projects that students could be expected 
to master themselves, or organized programs to recognize students' individual 
accomplishments. 

Student attendance. Several measures of school attendance were available based on the 
two student cohons in Portland. For the 1989-90 cohort in 1989-90 (i.e., compared to the 
1988-89 baseline year), there was no evidence that the treatment group had performed any 
better than the comparison group; in fact, the comparison group outperformed the treatment 
group on this measure. Students in both the treatment and comparison groups were absent 
more often during the 1989-90 school year than during the baseline year, following a trend 
seen at the other middle school sites. The percentage increase in mean percent days absent 
was 57% for treatment students (or about 6 days absent) and 38% for control students (or 
about 3 days absent); in general, the control group had significantly fewer absences in both 
the baseline year and during the first-foUow-up period. Although increased attendance was 
an aim of the project, it does not appear to have been achieved during the initial period of this 
evaluation (i.e., in 1989-90). 

Perhaps more significantiy, however, since follow-up services provided by the project 
continued to be provided to many 1989-90 students at George Middle School in 1990-91, 
the 1989-90 cohort of project participants outperformed the control group on attendance 
based on comparisons of 1990-91 minus 1988-89 absence rates. Specifically, comparisons 
of the differences in absence rates between the treatment and control groups for 1990-91 
minus 1988-89 revealed a statistically significant difference favoring the students who 
received project services. These 1989-90 treatment students recorded a gain in percent days 
absent of 1.8% from 1988-89 to 1990-91 (i.e., an increase of about one-third of the baseline 
standard deviation for the group), as compared to a gain of 6.1% for the comparison students 
over the same period (i.e., an increase of more than one baseline standard deviatiorj). 

These data for the first- and second-follow-up periods suggest attendance patterns for 
the two 1989-90 samples that, while showing increases in absences generally compared to 
the baseline period, also indicate the possible positive effect (or cumulative effect) of the 
GMS treatment beginning in 1990-91. While the comparison students' rates of absence rise 
steadily during both periods (in a pattern that characi irizes student attendance rates in both 
the lower and the higher grades), GMS students in the demonstration began to reverse the 
trend in 1990-91. 

For the 1990-91 cohort, the treatment group succeeded in outperforming the 
comparison students in both their first and second years of project involvement. By the end 
of tiie first-follow-up period, i.e., the 1990-91 school year, project participants recorded a 
gain in percent days absent of 2.2% over the baseline year as compared to a gain of 5% for 
the comparison group, a difference that is statistically significant. By the end of Uie second- 
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follow-up period (i.e., 1991-92), the treatment students had decreased their absence rate by 
1.9% compared to the baseline year, while the rate for the comparison students at that time 
had increased about .3% over the first-follow-up period and 5.3% over the baseline period. 
These results, combined with the 1990-91 result for the H ' 90 student cohon, are 
suggestive at least of t: positive improvement in the demonstration's effectiveness at 
combating absenteeism during 1990-91. Since, as was noted earlier, there was little formal 
attendance monitoring in 1990-91, one possible reason for this improvement was a 
perceptible difference in the extent to which the project reached out to students and sought to 
build rapport among project participants and other students on campus. In 1990-91, for 
example, the scope of project activity increased beyond drug and alcohol abuse counseling, 
due for the most part to the personalities and individual styles of the project's counseling 
team. At the same time, it is possible to speculate that, based on the patterns of these two- 
year attendance figures for the two cohorts, that two years of the sort of intervention 
conducted in Portland are needed before students might be counted on to become more 
engaged in school. 

In considering these attendance data, it is useful to be aware of the response-rates, or 
rates of data availability, for the two cohorts over time. For example, first-follow-up 
comparisons for the 1989-90 cohort were based on 86 and 89 students, respectively, while 
second-follow-up comparisons were based on 70 and 64 students; that is, over time, the 
number of cases in both the 1989-90 treatment and comparison groups with complete data on 
baseline and current-year absences decreased (e.g., as a result of student moves). Similarly, 
for the 1990-91 cohort, first-follow-up comparisons were based on 97 and 86 cases, 
respectively, while second-follow-up comparisons were based on 90 and 75 cases. The actual 
effects these rates of data availability may have on the results obtained cannot be known. 
However, adjustments were made to the data to estimate what the results would have been if 
100% response had been obtained at each follow-up point, and analyses based on these 
adjusted data show the same patterns and significant results as reported above (see Appendix 
D for information on how these data adjustments were made). 

In addition to data on attendance from school records, we collected some related data 
via survey. For the 1989-90 cohort in Spring 1990, we asked students whether they felt they 
had improved in getting to class on time as compared to the previous academic year. The 
comparison of these data for project participants and control students revealed no statistically 
significant difference between the groups. For the 1990-91 cohort, we asked students to 
evaluate their performance at getting to class on time once at the beginning of the 1990-91 
year and twice more — at the end of 1990-91 and, again, at the end of 1991-92. Subtracting 
the earlier ratings from the later ones and comparing the differences (i.e., the differences in 
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perceived performance), we (again) found no significant differences between the samples at 
any of the time-points. 

Student academic performance. Data for the 1989-90 and 1990-91 student cohorts in 
Portland are less convincing of positive effects due to the activities at CMS. The grade 
averages for the treatment students in 1989-90 increased by .43 (on a four-point scale); 
however, the averages for the comparison group also increased by .32 during this period. The 
averages for both 1989-90 samples in 1990-91 decreased, with the average for the treatment 
students falling back to their baseline level. Since the project did not work direcdy to 
improve students' academic performance, these results would seem to suggest that promoting 
student engagement, by itself, may not lead necessarily to higher grades. The rates of data 
availability on which these findings for the 1989-90 student cohort are based are reasonably 
high — over 90% for the first-foUow-up period and close to this level for the second-follow- 
up period. 

For the 1990-91 cohort, treatment students increased their grade averages by only .02 
over the baseline year, however, the grade averages in 1990-91 of control students declined 
by .24 during the period, resulting in a significant difference between the groups favoring the 
treatment students. By the end of the 1991-92 school year, the averages for both groups had 
declined sharply from the baseline — down about .70 for both groups. Once again, the rates of 
data availability on which these estimates are based were quite high, exceeding 85-90% in 
every case. 

In addition to grade averages, we also collected student survey information related to 
participating in class projects and doing homework, activities that might be expected to 
contribute to improved grades. For both the 1989-90 and 1990-91 cohorts, however, these 
data reveal no pattern of perceived improvement, a result that is consistent with the findings 
for grade averages reported above. 

Student persistence to graduation. Virtually complete data for the 1989-90 student 
cohort were available to examine the comparative dropout rates for treatment and control 
samples for the years 1989-90 through 1991-92. For each of these years, (perhaps because 
the students were below the minimum age at which they might opt to leave school) no 
statistically significant differences were found in the rates of dropping out between the two 
samples. When these rates were based on moved and expelled students as well as school- 
classified dropouts, there remained no evidence of difference between the groups. 
Information on the dropout rates of the 1990-91 student samples in Portland also revealed no 
statistically significant differences between the groups, in this case, for either the 1990-91 or 
1991-92 school years. 

The inclusive dropout rates for the 1989-90 student samples by the end of the 1991-92 
school year were 13.8% and 12.9%, respectively, for the treatment and comparison groups. 
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These same rates for the 1990-91 student samples by the end of the 1991-92 school year 
were 4.1% for both groups. At no time for either the 1989-90 or 1990-91 cohort did the 
dropout-only rate exceed 6.5% for either GMS or comparison students. 

Other flndings. In addition to these key findings related to areas of plausible 
achievements for the Portland demonstration, it is also worth noting that, for the 1990-91 
student cohort at the site, there is evidence that, in 1990-91, students participating in the 
demonstration were more likely to report increases in their parents' attendance at school 
meetings and, as previously noted, in their own discussions with counselors about drug and 
alcohol abuse. The first of these results is interesting and difficult to explain, since our 
implementation visits left us with the impression that encouraging increased parental 
participation was not likely to be an area of success for the project. By contrast, student 
reports of increased discussions with counselors regarding substance abuse was a result that 
was clearly expected from the nature of the intervention. 

Factors affecting realization of outcomes. The personnel changes that were made at 
the site beginning with the 1990-91 academic year may have tangibly changed the 
intervention. Specifically, the CSD caseworker who began in 1990-91 worked especially 
well with middle school students, encouraging their confidences and providing them with 
personalized care and attention. On our visits to the site, students were quite literally always 
around this staff person, and they seemed to look forward to talking with her and to sharing 
their thoughts and ideas with her during breaks and in walks around the campus as well as in 
class. She was frequently approached by students who were experiencing problems with a 
teacher or with classwork in particular subjects and asked to help by talking with the teachers 
involved. The new drug and alcohol speciaUst assigned in 1990-91 was much less the 
director of site operations than her predecessor had been, a situation that promoted more 
equal status between herself and the CSD caseworker and seemed to downplay somewhat the 
over-arching theme of substance abuse prevention. In short, the primary activity 
characterizing the project in 1990-91 and in 1991-92 was personal counseling, delivered 
informally as well as in more formal sessions, and a sense of caring and advocacy for 
students. 
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The Model School Adjustment Program 
Broward County, Florida , 

Overview of Project Activities, 1989-90 

The most immediate aim of the Model School Adjustment Program (MSAP) was to 
improve students* self-esteem. To the extent this aim was realized, it was expected that 
student attitudes toward school would improve, which, in turn, should improve attendance. 
Better attendance coupled with improved attitudes should lead to improved academic 
achievement. 

MSAP provided one semester of pull-out peer tutoring and individual and group 
counseling for 6th-grade students identified by referrals from feeder elementary schools. 
Students who completed the program were eligible for as-needed follow-up assistance (by 
means of pull-out from regular class or after school) throughout their enrollment at n\iddle 
school. MSAP also provided weekly sessions for parents on parenting skills and behaviors 
during the semester period, as well as family counseling services when needed. 

The program was developed and first implemented in one middle school in Broward 
County in 1985; with SDDAP funds it was replicated by the district in three additional 
middle schools. Although evaluation staff visited the original MSAP site on several 
occasions throughout the study period, to meet with project staff and to observe the program, 
this evaluation focused on the performance of MSAP and comparison students at the three 
MSAP replication sites. As is discussed later in this report, the amount and intensity of the 
interactions between the organizers of MSAP and staff at the replication sites was 
surprisingly limited given the geographic proximity of the sites within Broward County. 

Project staff. The Model School Adjustment Program was operated by two school staff 
members at each site — an instructional supervisor and a senior counselor The instructional 
supervisor trained and supervised the tutors, oversaw the tutoring sessions, and provided 
additional help to the project participants on academic matters. The senior counselor 
conducted the weekly group counseling sessions for the parents, oversaw and participated in 
the group counseling sessions with students, and provided additional counseling to the 
students and parents as needed or requested. 

Participant selection. The students who were served were selected from incoming 6th- 
grade classes according to several criteria: their academic performance must have been poor, 
despite IQs of at least 85; they must have exhibited behavior problems; or they must have 
been over age for their grade. The students who were identified for participation must (then) 
have volunteered for the project, and their parents must have agreed to attend the weekly 
parenting sessions throughout the semester — failure to attend on a regular basis would have 



lead to a student's dismissal from the program. The final project group selected for any 
semester was thus composed of those 30-35 students who were the first to get their 
applications in, signed by both themselves and their parent or parents. 

This selection process seems designed to produce project groups composed of the most 
motivated and conscientious of the nominated students. In fact, one interviewed instructiona! 
supervisor described it as a defect of the program that the neediest students might not get 
served. For the three sites implementing the MSAP model in this evaluation. Black students 
were most often found among the recipients of program services (i.e., an average of about 
60% of the total participants were Black). 

Project services. MSAP operated as a pull-out program, with students receiving 
approximately five hours of service per week — one hour per day — for 15 weeks. On four 
days, students received peer tutoring on assignments for their regular classrooms — material 
in which they were experiencing difficulties; on the fifth day, they received individual and 
group counseling. 

MSAP peer tutors were 7th- and 8th-graders who received credit for this elective class 
and were trained and supervised by the instructional supervisor who provided additional 
academic help to students. Tutors might have been former MSAP panicipants themselves, 
but most had not taken part in the program. At the end of each tutoring session, the tutor and 
the instructional supervisor signed a record sheet testifying that the student had completed 
the assigned work. That record sheet was then passed on to the regular classroom teacher 
who also signed it if he/she was satisfied that the work had been completed satisfactorily. 
Periodically, the record sheet was sent home with the student to keep parents informed of the 
progress being made. When they reviewed the sheet, they too signed it and returned it to the 
instructional supervisor. 

Parents also had to sign students' attendance and class participation logs each night, and 
they were encouraged to visit with MSAP teachers and counselors at the school. (Counselors 
also provided outreach services to families who found they were unable to make these school 
visits.) Project participants were awarded points for successfully completing assignments, 
and these points could be spent for pens, folders, and other school supplies. At the end of 
each semester, the 10 or so students who earned the highest numbers of points were rewarded 
with a field trip (to Disney World, for example). 

Individual, group, and family counseling services were provided by the program 
counselor, who also conducted the weekly parent groups. Persons interviewed at all four sites 
(including the original MSAP site) expressed the opinion that the parent counseling 
component was the most critical to the project's success. It was heavily focused on sound 
parenting skills and behaviors, and it was observed to be notably successful in getting and 
holding the attention of participating parents. 
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Notable Changes in Operations During 1990-91 

Site visits to the three SDDAP-funded MSAP sites late in 1990-91 and interviews with 
staff and students revealed few changes in the original design and strategies of the program; 
the quality of services delivered to participants seemed generally high. Our evaluation of the 
quality of the implementation in 1990-91 found three areas in which improvements would 
seem desirable: (1) more attention to involving the supervising teacher and the MSAP 
counselor in the daily tutoring activities, (2) more attention to activities that would introduce 
MSAP strategies into regular classes, and (3) more attention to the needs of students who 
may be the most at risk of educational failure and, as a result, the least motivated to enroll in 
the program. In addition, we distinguished between the MSAP's strategies to increase parent 
participation (e.g., regular communications with parents to keep them informed about project 
activities and accomplishments) and those activities aimed at involving parents in program 
decision-making (e.g., as members of advisory panels), and we found that while participation 
was encouraged strongly, decision-making involvement was much less of a priority. Finally, 
although the three replications were in many respects carbon-copies of the original program, 
what seemed to be lacking was the sense of integration on campus of the MSAP activity. 
Whereas at the site where the program was originally developed the principal paid special, 
daily attention to all program staff and students (including tutors), the replications seemed 
isolated from their schools and much less a point of particular personal pride for their 
principals. (Further discussion of this point with suggestion and possible causative factors is 
presented in Chapter 6 in the section on replicating dropout prevention programs). 

Follow-up services at the high school level for MSAP participants were in the process of 
being implemented at the original site of the program and, to a lesser extent, at the replication 
sites. However, because the study cohorts for this evaluation had not yet left middle school, 
any impact of these new service arrangements would not have been evident during the 
evaluation period. Somewhat ironically, as these service extensions were being put in place 
at the district level, principals at the three middle school replication sites indicated that their 
MSAP staff might have to be re-assigned following the 1990-91 academic year due to state- 
mandated school budget reductions. 

Status in 1991-92 

The Model School Adjustment Program (MSAP) received new SDDAP funding, and 
those funds, plus local funds, allowed the program to offer expanded ser.'ices. A coordinator 
was added at the district level, and three additional staff members served the six high schools 
in the county, monitoring the progress of MSAP program participants and providing services 
on a case-management basis. The three middle schools that were involved in this evaluation 
expanded their articulation, counseling, and identification relationships with 20 additional 
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elementary schools, and peer counseling at the middle schools was also expanded. The 
district program director reported that the political atmosphere in the state has been an 
important factor affecting the project. Specifically, the fact that state funding in Florida is 
based on the numbers of classroom teachers precludes the use of such funds for a pull-out 
program like MSAP; consequently, the support of local school boards (and federal grants) is 
critical. According to the program director, the success of MSAP led to the establishment in 
1991-92 of an ombudsman position within the district to facilitate students' transition from 
middle to high school. 

Project Outcomes 

The evaluation of the MSAP program has involved four student samples. Two samples 
of 88 and 71 students were drawn in 1989-90, with treatment students selected at random 
from the three MSAP replications funded by the SDDAP and control students selected from 
among 23 other middle schools in Broward County where the MSAP program was not 
available. These 1989-90 MSAP participants were in the treatment during the first semester 
of 1989-90, with follow-up activities continuing throughout that year; for this reason, all 
comparisons (i.e., first- and second-follow-up results compared to baseline data) utilize data 
for the 1988-89, 1989-90, and 1990-91 school years. 

Two samples of 91 and 1 14 students also were drawn in 1990-91, with the treatment 
students once again selected firom among the first-time participants in the program at the 
three SDDAP-funded sites. In this case, however, treatment students were selected from 
among the second semester participants in the MSAP. Analyses of first-follow-up minus 
baseline differences in absence rates and grade averages were therefore carried out two ways: 
first, 1990-91 school year data were used as the first-follow-up estimates and compared to 
1989-90 baseline measures; second, second semester 1990-91 data were used as the first- 
follow-up estimates and compared to the 1990-91 first semester measures that serve, in this 
case, as the baseline. For the 1990-91 student cohort, comparison students were drawn firom 
(only) two middle schools not offering MSAP services. 

Analyses of the data for the 1989-90 student cohort revealed substantial baseline 
differences between the treatment and control groups on academic performance measures 
(e.g., mean yearly grade averages of 2.37 for MSAP students compared to .72 for comparison 
students). These differences were traced to an error in the locally administered procedure to 
identify comparison students; specifically, too stringent at-risk criteria were applied to 
identification of the pool of comparison students, which, in addition to producing non- 
comparable samples, also necessitated data collections involving the 23 school sites, with the 
result (and additional problem) that too few surveys were completed with comparison 
students (i.e., 23 out of 71 students, or a 32% response rate). For this reason, many 
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comparisons between the 1989-90 student samples could not be made at all, and those that 
were made and are described below should be interpreted with the greatest caution. Because 
of the sampling problems and the extent of non-response among the comparison sample in 
1989-90, no adjustments were made to these data to estimate the results that might have been 
obtained with a complete data-set. 

In the selection of comparison students for the 1990-91 student cohort, this specific 
problem related to student identification was corrected. In addition, because comparison 
students were drawn from only two schools in 1990-91, the survey response rates for these 
students approached 75%. No statistically significant differences between the treatment and 
comparison group student samples were evident in comparisons of 1989-90 (i.e., baseline) 
absences and grades for the 1990-91 student cohort. In addition, comparisons of grade 
averages and percent days absent during the first semester of 1990-91 for these samples (also 
a baseline period, as the MSAP program for the treatment sample did not commence until the 
second semester of 1990-91) revealed no statistically significant differences. 

The issue of student self-selection into the MSAP and the potential biases resulting from 
this selection process on sample characteristics remained a concern in 1990-91, however. To 
examine the sorts of biases that might differentiate the treatment and control samples, we 
assisted the Broward County school district in surveying students in these samples to learn 
more about their motivations for participating in a program such as the MSAP. For treatment 
students, questions referred to the MSAP; for comparison students, the survey asked why 
they might decide to volunteer for participation in an MSAP-like program. Survey items 
probed students' sense of the support they would receive from their parents and their peers 
for participating, and they were asked to consider how much this type of program would help 
students (in general) feel a part of the school, do better in school, and, more specifically, help 
them to do better in school. As noted in Chapter 1, however, caution must be used in 
interpreting these data, since comparison student responses were based on " what if " 
scenarios while MSAP students were able to report based on their actual experiences with the 
program. 

In general, the clearest and most consistent pattern of differences between the 1990-91 
student samples from this motivational survey emerged from responses concerning how 
much the program(s) might help students. Students participating in the MSAP were more 
likely to report that an MSAP program can help students (in general) a lot to feel a part of the 
school (63% versus 41%), help students a lot to do better in school (74% versus 56%), and 
help themselves (i.e., the student-respondents) a lot to do better in school (71% versus 50%). 
The interpretation of any differences between the treatment and control samples for 1990-91 
that favor the MSAP students must thus at least consider the possibility that the results are 
due, at least in part, to students' dispositions prior to the start of the treatment; that is, that the 



attitudes that motivated students to enroll in the MSAP were themselves powerful forces for 
improved school performance. 

In 1989-90, short-term accomplishments were projected for the MSAP in three areas: 
increased parental support, increased student self-esteem, and improved student 
attendance and academic achievement. A longer-term accomplishment was seen as the 
MSAP's ability to reduce the dropout rate. In 1990-91, we also considered plausible 
increases in parents' attendance at school meetings, frequency of communications with 
teachers, and greater attention to schoolwork. In addition, we saw gains in students' self- 
esteem as a distinct possibility and felt that reductions in absences and gains in grade 
averages were likely. Data to address outcomes related to parental support and student self- 
esteem are provided by the 1990-91 student survey; insufficient survey data on these issues 
were gathered in 1989-90. Student attendance, academic performance, and persistence in 
school are addressed with records data for both 1989-90 and 1990-91 and survey data for 
1990-91. 

Parental support. In 1989-90 based on a 63% response rate, close to 40% of MSAP 
students reported that their parents had visited their classes at least once since the beginning 
of the 1989-90 school year, and about 80% of these students reported that their parents had 
talked with their teachers at least once since the start of school. Sixty-five percent of these 
treatment students reported their parents often checked homework. 

In 1990-91 and 1991-92, the student surveys administered to both the 1990-91 MSAP 
students and to the non-MSAP sample for that year allowed comparisons of parental and 
other (non-school) adult support across eighteen dimensions and two time periods. Ranging 
from parents' attending meetings at the school, checking of homework, and holding high 
educational expectations for their children to other adults providing help with class work and 
helping with student personal problems, these dimensions covered several of the areas in 
which MSAP parent training sessions and parental requirements for program support aimed 
their efforts. On none of these dimensions, based either on the first or second follow-up of 
students, however, did MSAP participants evidence any greater positive changes, or gains, 
than did the comparison students. 

These results may suggest that baseline differences between the samples on these 
measures acted to mask improvements — i.e., that MSAP students, if they were already 
reporting high degrees of parental involvement prior to the start of the program would have 
little room to improve. There is, in fact, only slight indication that MSAP participants 
differed significantly from control students on measures of parental and other adult 
involvement prior to the start of the program. Baseline differences on two of the eleven 
indicators of parental involvenent favored MSAP participants (i.e., measures related to 
parent attendance at school meetings and checking of homework), and none of the eight 
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indicators of other adult involvement showed any significant differences between the groups. 
In addition, results from the surveys that were intende'' to probe for factors underlying self- 
selection into the program indicated little differences the extent of presumed parental 
support for participation prior to the start of the treatment. Taken together, these data suggest 
that the lack of improved parent behaviors relative to school concerns among MSAP students 
is less likely to be due to their baseline status or disposition and, is perhaps, more a function 
of either the lack of sensitivity of the measures employed in this evaluation or the lack of 
effectiveness of the various MSAP parental involvement strategies. It is possible, for 
example, that students narrowly defined such survey items as parent attendance at school 
meetings to exclude any MSAP-related parent sessions at the schools that were components 
of the program. 

Student self-esteem. Comparing the mean composite scores on the Rosenberg Self- 
Esteem Scale of MSAP students and comparison students based on 1989-90 survey data, we 
find no statistically significant difference between the groups. These data were collected only 
once in 1989-90, as noted above, so no account could be taken of baseline differences 
between the groups in determining whether gains in self-esteem had indeed been achieved. In 
addition, as noted previously, the response rate to the survey among comparison students in 
1989-90 was low, thereby casting considerable doubt as to the representativeness of the 
results. 

Data from the student surveys administered to the 1990-91 student cohort also 
evidenced no differences in the gains in self-esteem made by the MSAP students as 
compared to their counterparts at non-MSAP schools. There were also no baseline 
differences between the groups on this measure. Our observations of the programs at the 
three SDDAP-supported sites in 1990-91 suggested that greater attention was being paid at 
these sites to helping students with class work and with personal problems than was directed 
specifically at self-esteem development activities (e.g., providing tangible rewards or prizes 
for particular achievements). 

Student attendance. In 1989-90, based on data availability for more than 71% of 
MSAP students and from 43% to 65% of comparison students, percent days absent for 
MSAP students had increased by 2.3% over baseline estimates; by 1990-91, this increase 
had reached 5%. Keeping in mind the problems with the comparison group used in 1989-90, 
we found no significant, relative improvements in attendance for MSAP participants in our 
1989-90 cohort either for 1989-90 (the year of their program experience) or for 1990-91 
(the year following their direct program experience). It was also the case that no significant 
improvements were found for MSAP participants in our 1990-91 student cohort for either 
the 1990-91 or 1991-92 school years; in these cases, MSAP participants increased their 
absences by 1.7% in 1990-91 relative to the 1989-90 baseline year, as compared to a 2.2% 
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increase for comparison students over the same period. In 1991-92 as compared to the 
baseline year, MSAP student absence rates had increased by 5.4% and comparison student 
rates increased by 4.6%. Data availability on absences for the MSAP students in 1990-91 
and 1991-92 exceeded 70%; for comparison students, the rate of data availability was 76% 
in 1990-91 and 64% in 1991-92. 

Because the initial MSAP treatment was for only one semester, and in the case of the 
1990-91 student cohort the MSAP treatment period was during the second semester of the 
school year, we also compared differences in absences rates for the two student samples 
between the first and second semesters of 1990-91 and between the first semester of 1990- 
91 and 1991-92. Our findings from uhese analyses also evidence no statistically significant 
differences between the groups. Specifically, MSAP participants increased their rates of 
absence during the treatment semester by 1 .3% on average over their rates for the first 
semester, compared to an increase of about 2.5% for the comparison students over the same 
period. (The pooled standard deviation for these estimates was just about 6%.) By the first 
semester of 1991-92, MSAP student absences had increased by 3.9% as compared to 3.2% 
for the comparison students, with a pooled student deviation of about 7%. Since attendance 
was a particular focus of the MSAP semester, with teachers and parents required to closely 
monitor student attendance on. a daily basis, this result is surprising. As noted above, no 
significant baseline differences were found between the 1990-91 treatment and comparison 
group samples (either in 1989-90 or in the first semester of 1990-91) that might help to 
explain these results. 

Student academic performance. Comparisons of grade averages for 1989-90 and 
1990-91 involving the 1989-90 student cohort showed the control group outperforming the 
treatment group for both periods. Problems with the selection of the comparison group in 
1989-90, however, make these results very difficult to interpret; for example, as noted above 
in the brief description of the evaluation design for the site in 1989-90, the baseline grade 
averages for these groups differed by more than one and one-half grade point favoring the 
treatment students. 

Also described previously, data for the 1990-91 student cohort revealed no baseline 
differences between MSAP participants and the comparison students. In addition, these data 
showed no differences in the 1990-91 minus 1989-90 and the 1991-92 minus 1989-90 
grade averages for these samples; for both groups for both time periods, grade averages 
decreased — by .26 and .45 for MSAP students and by .21 and .30 for comparison students. 
Similarly, there were no differences in the comparisons of current-year minus previous-year 
student self -reports of progress in work on class projects or in doing their homework for 
either 1990-91 or 1991-92. When we examine second semester 1990-91 minus first 
semester 1990-91 grade averages, we also do not find any statistically significant 

110 



ERIC 



differences. Specifically, MSAP participants raised their grade averages by about .07 during 
the treatment semester, while the comparison students' averages decreased by about .05 
during the same period (with a pooled standard deviation of about .44). Similarly, when we 
compare 1991-92 first-semester averages to first- semester 1990-91 averages, we find the 
MSAP averages decreasing by .10 and the comparison student averages increasing by 0.1. In 
sum, the MSAP effort in the second semester of 1990-91 produced no statistically 
significant, positive improvements in the performance — real or perceived — of the student- 
participants. 

Student persistence to graduation. Dropout rates at the Broward County treatment and 
control sites have been low over all the years and for the two cohorts involved in the 
evaluation. Considering school-classified dropouts only, the dropout rates for MSAP 
participants in the 1989-90 cohort were 2.3% in 1989-90 and 0% in both 1990-91 and 
1991-92; that is, while 2.3% of the students were listed as having dropped out in 1989-90, 
these students had re-enrolled and completed the school year in 1990-91, returning as 
enrolled students in 1991-92. Neither of these rates was reliably different from the 
corresponding rates for the 1989-90 comparison sample, however. 

Looking at the 1990-91 student cohort in 1990-91, no school-classified dropouts, 
expelled students, or moved students (i.e., those who had moved without confirmed re- 
enrollments) were recorded for either of the samples. By 1991-92, no school-classified 
dropouts had yet been recorded, but 2.3% of the MSAP students had moved or been expelled 
without confirmation of re-enrollment, along with 4.6% of the comparison students. The 
rates between these groups in 1991-92 were not statistically significant. 

Other findings. Data from the first-follow-up of the 1990-91 cohort show control 
students were reporting greater parental attendance at school meetings. These students also 
reported that they more often were reading books or magazines and that their parents had 
more positive attitudes about reading in general than did MSAP students. Finally, 
comparison students in 1990-91 were more likely to report themselves as being involved in 
social activities at school than were MSAP participants, a fact that is particularly disturbing 
because social activity at school, such as yearbook participation, has been suggested as an 
important way for students to form bonds with their schools that may prevent later forms of 
disengagement (see, for example, Finn, 1989). By the second-follow-up in 1991-92, MSAP 
students were found to have formed more positive images of high school graduates than had 
the comparison students (as measured by a composite of several items describing 
characteristics of high school graduates). At the same time, by the second-follow-up 
comparison students remained more involved in school social activities and were more likely 
to report their parents had attended school meetings; according to these comparison students, 
a greater number of their parents thought, more generally, that school was important. These 
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findings follow no particular pattern; however, taken together with the lack of evidence of 
positive results for MSAP participants based on either records or survey data, they leave 
much room for question as to the effectiveness of this dropout prevention model. 

Factors affecting realization of outcomes. The MSAP replications at the three sites in 
Broward County seemed rather faithful to the model site in all respects but one — the role the 
program played on campus in the views of the building principals. Because the MSAP uses a 
pull-out approach and may therefore have to combat the unintentional stigma associated with 
being labeled a program participant (see Chapter 6), the "aura" of the program on campus 
likely is quite important. At the model site, it is unquestionable that the program is the 
centerpiece of the school, with the principal making several informal visits throughout the 
day to interact with teachers and students. At the replication sites, however, the MSAP 
programs were conducted as "one more special program," with little or no special principal 
attention. While the replications thus provide a test of the program's design, they do not 
provide a test of the design as implemented at the model site. The sort of principal leadership 
and commitment present at the model site with respect to the program is, in our view, a 
critical factor determining what outcomes for students are achieved. 
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Authentic School Program 
Denver, Colorado 



Overview of Project Activities, 1989-90 

The Denver Public Schools implemented distinctive dropout prevention activities in five 
schools with the support of SDDAP funds. The Authentic School Program was one of these 
components and served students in grades 6-8 who were selected at random to receive special 
services within Lake Middle School in Denver. Gold Team students were taught by a team 
consisting of five teachers (one of them Spanish-speaking) and one counselor, and their 
classes had lower-than-usual student-teacher ratios. Perhaps the most important aspect of the 
program, the Gold Team students stayed together and remained with the same teachers 
throughout their school day at Lake (with the exception of physical education). This 
approach was designed to give students and teachers a greater chance of developing close 
relationships. It also aimed to help students make the adjustment from the one-teacher 
classroom at the elementary level to the more complex organization of the middle schooi 

The five Gold Team teachers and one counselor worked as a team themselves, jointly 
planning daily and weekly curriculum schedules and specific instructional responsibilities. A 
thematic curriculum was used: the four core subjects (language arts, mathematics, science, 
and social studies) were integrated; a Spanish class provided cross-subject instruction; and an 
elective class featured opportunities for enrichment, application of knowledge, and additional 
academic assistance. Gold Team teachers taught all of these classes and were able to 
organize and lead field trips with students when they wished. The teacher team also handled 
all disciplinary actions involving Gold Team students. The counselor provided both group 
and individual counseling to students and sponsored a parent support group and conducted 
home visits. 

Project staff. The director of the overall SDDAP grant that provided support to the 
activities in five schools was located at the district office. The assistant principal at Lake 
Middle School, who, together with the school's principal, wrote the proposal to receive 
district funding remained at the site through the 1988-89 school year. She was, however, 
transferred to another middle school site in the district ratlier unexpectedly at the end of that 
school year. The Gold Team itself was comprised of five teachers and a counselor, with one 
teacher hired specifically because of her experience teaching 6th-grade students in 
elementary schools in the district. Each of the other five teams at the school, which followed 
the typical middle school class schedule, had only four teachers. The added teacher enabled 
the Gold Team to reduce their class sizes and provide instruction in Spanish. Of the five Gold 
Team teachers, only two remained after the first year, with two new members recruited from 



within the school and the fifth being hired from outside the school. By 1990-91, none of the 
teachers who had begun with the Gold Team in 1988-89 remained with the program. The 
reasons for the turnover in staff are complex and are discussed in detail in Chapter 6 under 
the topic of sustaining educational reform activities. Most important, even by 1989-90, 
important changes in the original design of the program had begun to occur. 

Participant selection. In 1989-90, Lake Middle School served a disadvantaged 
population, 75% of which was Hispanic. Selection of students for the Gold Team in 1988-89 
was made at random from among the school's 6th-graders to facilitate evaluation of project 
activities. In 1988-89, 6th-graders were enrolled on the Lake Middle School campus for the 
first time. 

Project services. Gold Team teachers were able to provide students with rather 
intensive services. As noted above. Gold Team students received instruction from the team 
teachers for all but one period of the school day. In addition to being in smaller classes. Gold 
Team students benefited from several other services. First, Gold Team classes lasted for one 
hour instead of 45 minutes. As a result. Gold Team students were on a rotating schedule in 
which they took four of five academic classes on a daily basis. While on average the hours of 
instruction in each subject remained equal to those of other students. Gold Team teachers 
were able to extend classes as needed to enhance learning opportunities. 

Gold Team teachers handled all discipline problems as a group rather than sending 
students to the assistant principal for disciplinary action. This enabled them to tailor 
disciplinary actions to the particular personalities of the students involved and to concentrate 
on resolving problems rather than punishing behaviors. Gold Team teachers also designed an 
alternative grading system in which students were not assigned failing grades. Instead, 
students who failed a course received a No grade and a chance to do make-up work to 
change the NG to a C (i.e., grades of A or B were not allowed in making up an NG). To 
motivate students to take this opportunity to improve their grades, students were removed 
fix)m all academic enrichment activities until a C grade was earned or until the end of the 
semester grading period. If an "NG" remained at the end of the period, students received a 
grade of F. This grading system was designed to minimize students' sense of failure and 
maximize their proficiency in key academic subjects. Gold Team teachers designed special 
enrichment classes, including calligraphy, video yearbook, chemistry, and others, for their 
students. Students with "No grades" used the enrichment time to do their make-up work. 

Using their grant funds, the Gold Team teachers also hired a part-time counselor for 
their students. The counselor divided her time between Lake and one other school, offering 
both group and individual counseling to Gold Team students. Counseling sessions dealt with 
such issues as resolving conflicts, controlling tempers, and dealing with dysfunctional home 
situations. The counselor also sponsored a parent support group and made home visits. 
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Notable Changes in Operations During 1990-1991 

At the outset of 1990-91, three teachers left the Gold Team staff and were replaced by 
other staff who noted that this turnover did create problems for some students. (It should be 
noted that in 1989-90 two teachers had also left and were replaced.) In 1990-9 1 also, the 
policy of not initially assigning grades below C was changed to permit such assignments. 
The rationale of the principal and the teacher team was that since the students would enter 
high school in the next fall they should be re-introduced to the normal gi-ading procedure. 

The counseling component of the Authentic School Program at Lake also changed in 
1990-91. Whereas in previous years the Gold Team had its own counselor, in 1990-91 the 
counselor position was eliminated. As a result, students had to avail themselves of the regular 
school counselor, and after-school programs and programs for the parents of Gold Team 
members were discontinued. Parent Forums, which provided parents with an opportunity to 
discuss various issues with Gold Team staff were continued in 1990-91, but teachers 
reported that the numbers of parents participating and the length of the sessions were much 
reduced from the earlier years of project operations. 

Finally, the bilingual instructional program was modified in 1990-91; contact time with 
students was reduced (i.e., instruction was no longer provided to all students every day for 
the entire year) and actual instruction in Spanish was combined with cultural and minority 
studies. 

Status in 1991-92 

The Gold Team program was discontinued in 1991-92, and the building principal 
accepted an assignment as principal at a Denver high school. While the Gold Team approach 
no longer existed at Lake in 1991-92, team organization was still reported to be in use at the 
middle school. In • ddition, the Gold Team program was reported to have prompted a closer 
look at restructuring by school officials and to have created a sense of higher expectation that 
did impact the overall organization of the school. The program also seemed to have had a 
lasting impact on the members of the Gold Team. One of the Gold Team teachers reported 
that in December 1991, after leaving Lake and going on to 13 different high schools in 
Denver and several outside Denver, the Gold Team, on their own initiative, had a reunion. Of 
the 1 10 Team members, 87 (79%) attended and had such a good time reminiscing that they 
reportedly will plan another reunion for the future. 

Project Outcomes 

The evaluation of Gold Team activities at Lake is based on two samples of 107 and 1 13 
students selected at random from the Gold and Red teams, respectively, at the site. These 
teams had themselves been randomly established in 1988-89 when project activities at Lake 
began. For this reason, the baseline year used in analyses of student performance data (i.e., 
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absences, grades, suspensions) is 1987-88. Whereas Gold Team members were exposed to 
team-teaching and flexible scheduling of subject matters, Red Team members were enrolled 
in the typical middle school program. 

Although the two teams reportedly had been randomly chosen, we did compare them on 
baseline measures of absence rates and grade averages. Based on available records data for 
67% and 78% of the students. Red Team students were absent fewer days in 1987-88 than 
were Gold Team students; however, the numbers of absences for both groups were low 
(i.e., 8 days and 4 days, respectively, for the Gold and Red Teams). No differences between 
the grade averages for the groups were found (i.e., the mean averages were 2.71 and 2.52, 
respectively). Since the students were in 5th grade during the baseline year, a time when 
suspensions from school are rarely given out as punishments, neither group recorded any 
suspensions during the period. 

In 1989-90, four areas of plausible student achievements were identified for the 
Authentic School Program at Lake. First, in the short term, we felt it was possible for Gold 
Team students to improve their attendance, their academic performance, and their self- 
esteem. Second, over the longer term, the project seemed to be likely to help in encouraging 
Gold Team students to persist towards graduation. As described above, however, the 
changes that were made in the program in 1990-91 would seem to have diminished the 
program's potential for achieving these sorts of positive student outcomes. In fact, as 
described below, it does seem that 1989-90 was the high point of the three-yeai project 
period, at least as measured in terms of student accomplishments. 

Student attendance. Data on student attendance revealed that Gold Team students 
outperformed the control group on attendance measures based on calculations of percent days 
absent in 1989-90 compared to baseline estimates. Although both groups increased their 
rates of absences over the baseline year (which is a rather typical pattern describing the 
experiences of students moving from the elementary to the middle school grades), Gold 
Team members recorded an increase in percent days absent of 2.4%, compared to a gain of 
5.4% for the comparison group. (Complete records data enabling these gain-score analyses 
were available for 69% and 64%, respectively, of the treatment and comparison group 
samples in 1989-90). There were no differences between the groups in their perceived 
performance in getting to class on time— a different but related measure of physical 
engagement in school activity. 

For the 1990-91 school year, there were no statistically significant differences between 
the absence records for the two groups of students; however, the general trend in these rates 
for 1990-91, when compared to the 1989-90 baseline year, seemed consistent with the 
findings for 1989-90 (i.e., gains of 2.9% and 5.8% were recorded in the absences rates for 
the GoiJ and Red Teams, respectively). These results were based on (complete) records data 
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available for 64% and 59% of the treatment and comparison samples and were consistent 
with results based on data adjusted for nonresponse. 

Student academic performance. Data on grade averages, collected for comparable 
percentages of students, indicated no significant differences between the treatment and 
comparison groups in either the 1989-90 or 1990-91 school years. This is interesting given 
the no-grade policy adopted by the Gold Team for a large portion of this two-year period. 
Evidently, encouraging students to make up grades lower than C by not initially assigning D 
or F grades and withholding enrichment activities until passing grades were earned was not 
sufficient to produce higher grades. It is always possible that the grading standards used by 
Gold and Red Team teachers differed consistently, with Red Team teachers typically giving 
higher grades; however this would seemingly have been the sort of practice that Gold Team 
teachers would have been most sensitive to and would have taken into account when devising 
their own grading system. 

Test data would have been particulariy useful for assessing gains in academic 
performance at this site, due to the fact that data on yearly grade averages were affected to 
some degree by the No Grade policy for Gold Team members. For 1989-90. only the grade 
averages of three students included NGs, however, and in these cases only one NG for each 
student was assigned and not made up, or left to revert to a D or an F. The influence of 
these made-up grades was probably not a significant factor affecting overall grade averages 
for the group. 

Survey data collected in 1989-90 (also) showed nothing to indicate that Gold Team 
members felt their perfonnance on class projects or in doing their homework had improved 
since the beginning of that year. 

Student self-esteem. The comparison of 1989-90 scores for the Gold and Red Teams 
on the Rosenberg Self-Esteem Scale show no statistically significant differences between the 
groups. Of course, without baseline information on self-esteem levels it is difficult to 
interpret these data. 

Student persistence to graduation. Dropout rates based on school-classified dropouts 
only remained low (i.e., less than 2%) for both the Gold and Red Teams throughout 1989-90 
and 1990-91. In 1991-92, the rates for these teams were 2.2% and 5.9%, respectively. When 
moved and expelled students are added to the totals for each year, the rates of dropout 
fluctuate over the years, increasing to between 1 1% and 12% in 1990-91 and decreasing to 
3.3% and 5.9% in 1991-92, but there are no statistically significant differences in the rates 
for the two groups. 

Other findings. During the 1989-90 school year. Gold Team students had significantly 
fewer suspensions than control students. By the 1989-90 school year (when the students 
were 7th-graders). Gold Team students had recorded a .06 average suspension rate compared 
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to a rate of about .32 for the Red Team students. This statistically-significant result probably 
reflected the family oriented approach of the Gold Team to handling discipline problems in 
that school year. At this site, team teachers assigned to these students were charged with 
deciding upon disciplinary actions, and their aim, in addition to providing some form of 
punishment, was to help build a sense of team pride. As a consequence, student behavior 
problems were discussed among team members, group decision-making processes often were 
used to address these problems, and students were encouraged to help their peers improve 
their attitudes and interactions. 

By 1990-91, the difference in suspensions between the Gold and the Red Teams 
disappeared; specifically, the number of suspensions among Gold Team students increased 
substantially over their level in 1989-90. Since as noted earlier many staffing and policy 
changes took place in 1990-91 that affected the Gold Team, it is possible that suspensions 
increased in 1990-91 either because the new teachers sought lo impose different and more 
strict rules for behavior or because the responsibility for disciplinary actions involving Gold 
Team members was transferred from the team teachers (back) to central school 
administrators. This may have occurred in some cases if not in all due to the termination of 
the Gold Team counselor position in 1 990-9 1 . This individual had been involved in handling 
disciplinary matters involving Gold Team students in 1989-90. 

1 he most consistent pattern of results for the Authentic School Program concerned 
student perceptions of the school climate in 1989-90. Three indicators comprised of the 
responses to several individual survey items concerning teacher attitudes, campus discipline, 
and school safety (item 25 on the SI survey form in Appendix E) revealed clear differences ' 
between the Gold and Red Teams. Specifically, based on 77% response rates from the two 
student samples, 82% of Gold Team students (vs. 54% of the control students) reported their 
teachers were more caring, 64% of the Gold Team students (vs. 45% of the control students) 
thought that the discipline at their "school" was more firm and fair, and 54% of the Gold 
Team students (vs. 35% of the control students) reponed that their "school" was safer or freer 
from unwarranted disruptions. These data argue strongly that the efforts made by Gold Team 
teachers to personalize the middle school experience and to make it more adaptable to 
students' learning-related needs and styles were having effects, at least when measured in 
1989-90. In that year as well, students exhibited a great deal of pride in being on the Gold 
Team. During our 1989-90 visit to the site, one student commented, "The Red Team's work 
is below ours," based on the homework he saw some of his Red Team friends doing. Gold 
Team students also said they were glad that they were going to have the same tei chers during 
the next year because "(Gold Team] teachers reason with you. They talk with you and help 
yoL,. Other teachers just give you a referral (to the assistant principal).'^ Finally, one student 
merely was grateful for the attention he received, saying, "If you're shy, other teachers don't 
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get to know you. But these get to know you." Considered together with the positive outcomes 
related to attendance and disciplinary actions found for 1989-90, the data related to school 
climate suggested the sorts of intermediate outcomes that may be of particular benefit to 
students who are at risk. 

Factors affecting realization of outcomes. The lack of positive findings following 
1989_90 is probably explained by the wholesale changes made to the program. In addition, 
the program was poorly introduced to other, non-participating faculty in 1988-89, which 
accounted for the good deal of resentment of the Gold Team that built steadily over the 
evaluation period. With the district's growing involvement in reducing the program budget 
and making staffing changes over the life of the demonstration, the project design as 
conceived was only implemented in 1989-90 (even though the designer of the program had 
already been transferred before the beginning of that year). 
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Tri-School Project 
San Antonio, Texas 



Overview of Project Activities, 1989-90 

The Tri-School Project in the North East Independent School District (N.E.I.S.D.) 
served students in two middle schools and in the high school into which those middle schools 
fed. The design of the project at the three sites involved small, pull-out classes, computer- 
assisted instruction (CAI), and individual counseling on an as-needed basis. The focus was 
clearly on building academic skills, and a clearly-defined project aim was to assist students in 
passing the mandated state tests (Texas Assessment of Academic Skills). Core classrooms, 
which housed the program's CAI learning stations, also featured opportunities for students to 
engage in cooperative learning strategies and peer tutoring. 

Project counselors were an integral part of the design; they were housed near the 
classrooms to encourage students to drop in and to separate the counseling function from the 
provision of disciplinary services. At the outset of the project, coordination among the three 
sites was to be emphasized; however, changes in the project director position militated 
against the development of programmatic linkages. 

Project staff. Each of the three schools had two full-time teachers and one full-time 
counselor for the project students. All three staff had worked in San Antonio schools for 
some time and chose to work with the population of students at risk served by the Tri-School 
Project. The overall project director was based in one-half of a portable classroom located on 
the Roosevelt High School campus. The role of the director was to monitor progress made by 
the project staff at each school site and to facilitate cross-site linkages (e.g., sharing of 
materials or strategies). For a variety of reasons, the project directorship changed a number of 
times during the funding period. For example, the first three directors chose permanent 
employment in the district rather than waiting to see whether funds would be available in 
1990-91 for continuing the project. In contrast, little or no turnover occurred among the 
teaching and counseling staff. Project staff at the three schools maintained close contact with 
the regular teaching staff who had the students for al! other classes, informing them about 
students' progress and learning about students' behavioral and academic progress in their 
regular classrooms. 

Participant selection. Students were selected by counselors and project staff according 
to the state at-risk criteria. These criteria were extensive, covering a diverse range of at-risk 
indicators, including SES, parent-household status, family dropout history, AFDC eligibility, 
truancy, and so on. The piirents of those most at risk, based on ratings on each at-risk 
criterion, were contacted first and told about the reasons for the program and the prospective 
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benefits to their chUdren of enrolling. The counselors or teachers also interviewed the 
potential participants to discuss the responsibilities students would have to the program and 
the ways that being in the program would help them. Throughout the year, new students 
could be selected to participate, although the attrition rate was very low and the project staff 
wanted to maintain small class sizes. Slightly under one-half of the students served were 
Hispanic, with white and Black students combined comprising the majority of program 
participants. 

Project services. About 35 students at Krueger Middle School (Target K) and 75 
students at White Middle School (Team Eagles) in grades 6-8 spent from one or two hours 
every day (depending on the grade level and student need) in pull-out, remedial classes that 
concentrated on math and language skills, using computer-based tutorials. The classwork at 
Roosevelt for the 40 9th- and lOth-graders was self-paced, with computer-assisted programs 
specifically designed to increase the numbers of students passing the mandated state tests. 
Project classes were small (i.e., 10-15 students), and every individual received personal 
attention daily from a certified teacher and an aide. No credit toward graduation was given 
for participation in Tri-School courses. 

After-school and Saturday tutoring was also provided at the three schools on an as- 
needed or requested basis. Various extracurricular activities were on-going, such as the 
Lunch Bunch students at Roosevelt High who met regularly for activities that included 
hearing guest speakers from the community and discussing teen issues. 

Project staff closely monitored student progress in other classes and met with non- 
project teachers to tailor project curricula to the content areas being covered in those classes. 
The counselors for the project were key to helping participants build positive interpersonal 
relationships and were skilled in referring individuals with extreme problems to community 
mental health agencies. The Tri-School teaching staff were trained in the use of new software 
and computer-assisted techniques, classroom management methods for students at risk, and 
strategies for developing self-discipline and conflict management skills in the students. 

Notable Changes During 1990-91 

A new director (the fourth since project inception) came to the Tri-School Project at the 
beginning of 1990-91. With a background in curriculum, this individual saw the need to 
broaden the instructional options available to students; in particular, she decided that the pull- 
out, tutorial approach should be supplemented by classes in substantive areas for which 
students earned credits toward graduation. At one of the middle schools, the two teachers 
offering the pull-out tutorial were split apart, and each one also taught classes in math, 
writing, and English for Tri-School project participants. One result of this change was that 
twice the number of students were served at the site by the project in 1990-91 than were 
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served in previous years (i.e., 150 rather than 78 students). At Roosevelt High School, the 
two pull-out teachers were re-assigned to teaching a total of six classes: three classes of 
Algebra and three classes of pre-Algebra. These classes were specifically for Tri-School 
project students and, for many, was their first opportunity to take any form of Algebra. In 
sum, 1990-91 was a transition year for the Tri-School project, from a pull-out, tutorial 
program offered as a special service to students to a regular classroom experience with lower 
student-teacher ratios and credits for students who successfully completed the work. 
Status in 1991-92 

Funds were not obtained from any source to keep the Tri-School Project operational. 
Texas state funding for school districts was cut in 1991-92, making it difficult for districts to 
fund programs locally. Funding for parental involvement programs was being pursued, 
however, as were funds for continuation of some aspects of the Tri-School instructional 
programs. 

Project Outcomes 

One hundred Tri-School students were selected at random from among the 150 middle 
and high school participants in the project in the fall of 1989-90. One hundred comparison 
students were selected from the waiting lists of candidates for the project at each of the three 
schools. The characteristics of the two pools of students were closely matched; the majority 
of students in both groups were Hispanic/Latino, came from lower socioeconomic status 
households, had failed at least two classes since enrollment at their school, had failed in at 
least two subparts of the TEAMS (now the Texas Assessment of Academic Skills), and were 
regarded as behavior or truancy problems. No baseline differences between the project and 
comparison students were found on any of the three indicators of school performance relied 
upon by the study: percent days absent, yearly grade averages, or total numbers of recorded 
(in-school and out-of-school) suspensions. For each of these indicators, records data were 
collected for about 75% or more of the sampled students— the one exception was 
suspensions data, which were available for only 52 of the comparison students. 

Improved rates of TEAMS test passage and improved attendance were judged to be 
possible short-term outcomes of the Tri-School effort. Improved academic performance and 
successful passage of the state test for graduation were identified as possible future 
outcomes. 

Clearly, the scores on the TEAMS test would be important indicators of the Tri-School 
project's success. TEAMS test scores were not available for students during the 1989-90 
school year, however (i.e., the year in which they began active participation in the project). In 
1990-91, some students were given the TEAMS test while others received a new test (the 
TAAS), which has now rcpl -^ed the TEAMS. Only students in grades 9 and 1 1 took one or 
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the other of these tests, and the scores for each of the tests were reportedly based on 
incompatible scales. For all of these reasons, we found it impossible to use the data from 
these test administrations as indicators of students* academic performance. Our assessment of 
improved performance in this area is based solely on students' grade averages and reports of 
improvements in work on class projects and homework. 

Student attendance. In contrast to the usual pattern for middle schools in this study, 
students in both of the San Antonio treatment and control groups improved their attendance 
rates slightly during the 1989-90 school year. For Tri-School participants, their rates of 
absence decreased by 1.2% on average, while comparison student averages decreased by 
.2%. These differences between the two groups were not statistically significant, however. 
Comparing 1990-91 absences rates to those in the baseline year, both groups of students on 
average recorded increases— Tri-School students an increase of 1.5% and comparison 
students an increase of 1%. These differences between the groups were also not statistically 
significant; however, they were based on very limited data. For example, records data were 
obtained for fewer than 50% of the comparison students and for only 59% of the Tri-School 
students. Gain-score analyses run on these indicators using the file adjusted for nonresponse 
also showed no statistically significant differences, but caution in interpretation must be 
advised due to the poor response rates on which these estimates are based. 

Students' evaluations of how well they were doing in getting to class on time, made in 
the spring of 1989-90, also showed no differences between the groups. Approximately 45% 
of the Tri-School students and 42% of the comparison students reported improvements in 
these behaviors as compared to the previous school year. 

Student academic performance. Comparisons of the grade averages of Tri-School 
students with those of control students revealed a statistically significant difference favoring 
Tri-School participants in 1989-90; by 1990-91 no difference between the groups was 
found, although the same caution pertaining to the interpretation of attendance data apply in 
this case also. By the end of 1989-90 (with complete records data available for all students), 
Tri-School students had increased their grade averages by about one-tenth point, (i.e., .096), 
while comparison students' averages decreased about one-fifth point (i.e., .195). By 1990- 
91, the grade averages for both groups had declined— comparison students by about .43 
relative to what they were in 1988-89, and the averages of Tri-School students by about .12 
from the baseline year. Records data for these 1990-91 minus 1988-89 comparisons were 
available for 61% and 46%, respectively, of the treatment and control students and, although 
the findings were the same based upon the imputed file (which estimated values for students 
with missing data), caution in interpretation is advised. 

There were no differences between the groups in self-reports of performance on class 
projects or in doing their homework compared to the previous year. 
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student persistence to graduation. Students from the 1989-90 student cohort selected 
at Roosevelt High School (one of the Tri-School sites) had yet to graduate by 1991-92, so it 
was not possible to estimate differential rates of passage of the state-mandated test for 
graduation. Dropout rates for the three sites comprising the Tri-School project also revealed 
little about the effectiveness of the Tri-School project in preventing dropout from school. 
With virtually complete data available, comparisons of 1989-90 and 1990-91 dropout rates 
between the Tri-School students and the control students produced no statistically significant 
findings, whether or not moved and expelled students were included in the estimates of 
dropout rates. The rates of school-classified dropouts only for the two groups over the years 
ranged from 4% to 9% for the Tri-School students and from 6% to 12% for the comparison 
students. The rates of dropout including moved and expelled students for these groups ranged 
ftiom 8% to 15% for Tri-School students and from 15% to 19% for the comparisons. 

By 1991-92, with data available for 97 out of 100 Tri-School students and 88 out of 100 
comparison students, the dropout-only rates for the two samples were 15% and 25%, 
respectively; the inclusive dropout rates were 20% and 38%, respectively. The difference 
between these 1991-92 inclusive rates are statistically significant while the former difference 
in rates (i.e., dropouts-only) is not. Although the general pattern of rates over the three years 
suggests that Tri-School students were somewhat less likely at the end of any one year to 
have dropped out (or moved or been expelled), the fact that in only one instance were the 
rates between the groups shown to be significantly different statistically suggests care must 
be taken to not over- interpret these data. 

Other findings. Suspensions increased during 1989-90 for both the Tri-School and 
control groups, with no significant differences in rates. The rates of suspensions in that year 
for both groups were high and just about doubled from the baseline year (average of about 
one suspension per student per year) to the end of the first-follow-up period. These data were 
considerably skewed, with some students receiving many suspensions and most receiving 
few if any suspensions. Examining the records of individual students, we found the reasons 
given for suspensions to be quite varied— e.g., from pencil-fighting and throwing away 
official school forms to not dressing out for physical education and being chronically tardy to 
school. For the 1990-91 school year, data on suspensions from school were simply not 
available for even the minimum numbers of students to permit meaningful analysis. 

No differences also were found between the groups on measures of parent involvement 
in school-related activities (e.g., attending school meetings, checking homework) or on 
student perceptions of the quality of their school climates. These findings on parental 
involvement are in contrast to reports from the site in 1990-91 that suggested parental 
involvement had grown, particularly at the two middle schools, and that adult participation 
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through monitoring, especially with the encouragement of the project's advisory board, had 
also increased at all the sites. 

Factors affecting realization of outcomes. The higher grades earned by Tri-School 
participants in 1989-90 may have been the result of intensive remediation with computer- 
assisted driU and practice. The fact these gains did not persist, however, may also be related 
to this approach; namely, the returns to students from this instructional strategy may have 
been mitigated by the negative labeling they experienced as a result of being pulled out for 
remedial classes for which no credit was provided. Because of the turnover in the director 
position, there would likely have been no on-site monitoring to prevent just this sort of 
problem. The positive result for Tri-School participants concerning dropout rates may have 
been stimulated by the close coordination between the middle and high school sites, but the 
case for coordination is weakened on account of the inconsistency of oversight (i.e., the 
project director position) at the demonstration. 
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Project Success, JHS #113 
Bronx, New York 



Overview of Project Activities, 1989-90 

In the Bronx, SDDAP funds were used to support demonstration activities at two middle 
school sites— JHS 113 and JHS 142. Data gathering was attempted at both sites for this 
evaluation; however the emphasis of the evaluation was placed on activities at JHS 1 13. The 
dropout prevention project at this site provided a variety of extra services to the entire 8th 
grade in the school. Program components included the addition of personnel to reduce class 
size, a council composed of staff and administrators to address student problems, an 
incentive/reward system for student achievement, and an after-school tutorial program in 
math and reading. The school was located in a middle income neighborhood in the northeast 
Bronx. It had 1,327 students in grades 7 through 9: 70% were black and 25% were Hispanic. 

Demonstration funds were also used to buy a variety of curriculum materials, including 
language tapes, high-interest reading materials, career books, and supplies. A newsletter on 
project activities and awards was published for parents. 

An after-school program for tutoring in reading and math was offered twice a week for 
two hours each day. 

Project staff. Project funds were used to hire three extra 8th-grade teachers. Which 
allowed average class sizes to be reduced from 26 to 22 students per classroom. The 8th- 
Grade Council, which was organized to plan activities and develop strategies to help keep 
students in school, was led by the assistant principal and included the school's guidance 
counselor, dean of student activities, 8th-grade teachers, and the SDDAP project coordinator. 
While the general charge of this council was to coordinate effons to help students adjust to 
school, its primary activity was to supervise the administration of section sheets (described 
below). 

The overall SDDAP project coordinator was a staff member at JHS 113 who supervised 
project activities at this site along with those at JHS 142. 

Participant selection. Because this was a grade-wide project, all students in the 8th 
grade (approximately 500 students) had access to project services.' Students were referred to 
or volunteered for the after-school tutorial portion of the project, and approximately 80 
students attended these sessions throughout the year. 

The after-school program provided assistance in math and reading skills primarily for 
recent Caribbean immigrant students, who made up a large percentage of the school 
population. Most participants in the program were black, with Hispanic students comprising 
about 20% of the population served. 
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Project services. Students received an average of 10 hours of service per week over the 
school year. An incentive program operated for all students through the accumulation of 
points on "section sheets." When good scores were accumulated, a variety of activities were 
offered as rewards. Every day, each classroom teacher graded the class on their attendance, 
behavior, and performance. Project staff believed that peer pressure provided an important 
source of motivation in the Bronx, and they noted that one student could very well ruin the 
chances for the whole class by inappropriate behavior on a given day. Score-cards were 
posted in the classes and in the halls. Field trips consisted of visits to recreation camps in the 
country, the circus, and other special places. Awards were also given to students who earned 
high academic averages and students who regularly attended the after-school program. 

Notable Changes in 1990-91 

The class-wide project continued, although a number of JHS 113 students had opted for 
transfers to the district's high schools. As students moved from 8th grade to the 9th at JHS 
1 13, the extra teaching personnel moved with them. 

Status in 1991-92 

With the termination of the SDDAP grant, the extra teacher positions that had been 
supponed were also terminated. Council activities were cunailed, owing to the lack of funds 
to suppon release- or planning time. The use of incentives was continued on a class-by-class 
basis; that is, incentives were no longer provided to students grade-wide. 

Project Outcomes 

For the treatment group, 1 10 students were selected at random from the 8th grade class 
at JHS 113; for the control group, four classrooms of students (i.e., 86 students in all) who 
were comparably at risk of dropping out as evaluated by the project coordinator were 
selected from another school. Both schools served mostly black and Hispanic students from 
lower socioeconomic levels, and the students at risk involved in the study were reportedly 
identified by the district on the basis of poor academic performance. 

Several problems were encountered in collecting evaluation data from the student 
samples at these New York schools. First, time was required to obtain formal Board of 
Education approval for the data collection, which delayed the start of records data collection. 
When Board approval was received, it was conditional on evaluator receipt of active parental 
consent for all study participants before records or survey data could be collected. Always 
difficult to achieve in a timely manner, active consent (i.e., written parental authorization for 
student participation) was further complicated by teacher decisions not to collect returned 
forms or to remind students that consent forms should be completed. This task fell to on-site 
evaluation staff who visited these Bronx schools on a daily basis for much of the winter and 
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early spring 1989-90. When federal authorization was received to begin survey data 
collection, surveys were administered to students who had returned consent forms. The 
response rate for the students at JHS 1 13 was well below 50% (i.e., less than one-half of 
those selected initially for the study had consent forms and responded to the survey), making 
these data suspect in terms of their representativeness both for analysis and as a basis for 
imputation prior to analysis. Records data collection, which was begun shortly after 
administration of the surveys, was severely hampered by the fact that the records for many 
students had been forwarded to high schools throughout the city by the time this phase of 
data collection could be initiated. As with the survey data, records data on absences and 
grades were collected on fewer than one-half of the treatment and control samples. 

The implementation study site visit in 1989-90 concluded that problems with project 
operations and the relative lack of intensity of the intervention suggested that school 
performance changes were likely to be infrequent. There was speculation, however, that 
project activities might have resulted in improved self-concepts among project participants. 
During the site visit, for example, students were observed to be enthusiastic about the award 
structure and the after-school tutoring program. 

As a result of the difficulties with data collection and the judgment that student-gains 
were unlikely, all evaluation activity at this site was discontinued in 1990-91. 
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Summary and Interpretation of Findings for 
Middle School Projects 

Summary of Findings 

Table 15 presents the summary of improvements for the four middle school projects 
with student samples participating in the in-depth evaluation. The findings in Table 14 are 
based on 1989-90 and 1990-91 student cohorts and summarize all positive findings for the 
years 1989-90 through 1991-92. As described previously, experiments are defined 
relatively; that is, an improvement is determined relative to a comparison group and, in the 
case of records data, improvements are also relative to baseline levels of performance. While 
the number of significant, positive findings is limited, the Denver-Lake and Portland sites 
seem to provide rather consistent evidence of gains for project participants — if only for a 
single period of the evaluation. (Note that data collected from the New York site in 1989-90 
were insufficient for inclusion in this table.) 

The data for Denver-Lake are noteworthy because they begin to confirm generalizations 
based on previous research regarding the relation of school climate to school participation. In 
particular for Denver-Lake, students participating in the SDDAP-supported demonstration 
were less likely to be absent in 1989-90 if they perceived their teachers as caring for them— 
e.g., being interested in students, willing to praise students for effort, able to listen to what 
students had to say — and students participating in the demonstration were absent less 
frequently and perceived their teachers to be more caring in that year. (The correlation 
between students' ratings of their teachers on these dimensions and percent days absent in 

1989- 90 was .23.) Since the demonstration aimed at altering the typical middle-school 
teacher-student relationship (e.g., by keeping teachers together with students for the entire 
day, permitting teachers the flexibility of team teaching and block scheduling of classes), 
these results may indicate some accomplishment of these aims. 

The correlations between two other aspects of school climate, discipline and campus 
safety, and absenteeism in 1989-90 were not statistically significant, and correlations 
between any of the indicators of school climate measured in 1989-90 and absenteeism in 

1990- 91 were not statistically significant. The rates of absences for the Denver-Lake 
treatment and comparison students in 1990-91, however, were almost the same as those in 
1989-90 (e.g., the correlation between 1989-90 and 1990-91 absences for treatment students 
was .70), and the 1991 minus 1989 difference in absence rates between the samples very 
nearly approached statistical significance (p = .08) favoring the treatment students. 
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Table 15 


Summary of Improvements for Students Served in 
the Four SDDAP Middle School Demonstration Projects: 


Measures Based on School Records and Student Self Reports^ 


Portland, OR 


Broward County, FL 


1989-90 Student Cohort 


1989-90 Student Cohort* 


First- Year Follow-Up 


First- Year Follow-Up 


• No improvements found 


• No improvements found 


Second- Year Follow-Up^ 


Second- Year Follow-Up'^ 




• No improvements found 


Third- Year Fol!ow-Up^ 


Third- Year Follow-Up^ 


• No improvements found 


• No improvements found | 


1QQA_Q1 CttaHont l^nlinrt 


1990-91 student Cohort 1 


First-Year Follow-Up 


First- Year Follow-Up | 


• Fewer absences (sr) 


• No improvements found 


• Higher grades (sr) 


Second- Year Follow-Up 


• More frequently talked to counsclor(s) 
about drug or alcohol vbuse (ss) 


• More positive image of high school 
graduates (ss) 


• Parent(s) attended school meetings (ss) 




1 Second- Year Follow-Up 




1 • Fewer absences (sr) 




1 Denver-Lake, CO 


San Antonio, TX 


1 1989-90 Student Cohort 


1989-90 Student Cohort 


1 First-Year Follow-Up 


First-Year Follow-Up 


1 • Fewer absences (sr) 


• Higher grades CsrJ 


1 • Fewer suspensions (sr) 


Second-Year Follow-Up^ 


• More caring teachers (ss) 


• No improvements found | 


• Fairer discipline (ss) 


Third- Year Follow-Up^ | 


Second- Year Follow-Up'^ 


• Lower dropout raie^ (sr) \ 


• No improvements found 




Third- Year Follow-Up^ 




• No improvements found 





^ All estimates of improvements for students in the demonstration projects are relative to comparison students; 
that is, means or rates assessed at one point in time or gain-scores for students in the demonstrations are 
determined to be improvements by comparing them to similar measures for comparison students. Absences, 
grades, suspensions, and dropout rates are based on school records (sr); all other measures are derived from 
student surveys (ss). Survey data for the 1989-90 student cohorts arc based only on one administration of the 
survey instrument — i.e., during the First- Year Follow-Up. Complete information for all Hndings is provided 
in Appendix A 
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2 Only possible improvements in absences, grades, and dropout rates were assessed for this follow-up. as 
student survey data were not available for analysis 

3 Only possible improvements in dropout rates were assessed for this follow-up. as neither student survey data 
nor other records data were available for analysis 

4 An effor in the matching procedure as implemcnicd at this site for selecUon of the 198^90 student cohort 
produced incomparable samples; extreme cauUon must be used in interpreting any results— i.e.. evidence of 
improvements or lack of improvements for students— based on this cohort 

5 This rate is based on school-classified dropouts together with moved and expelled students 



The findings for the Portland demonstration based solely on the 1989-90 student cohort 
are scant; that is. Table 15 only shows, that by 1990-91, students participating in this project 
had achieved lower rates of absences relative to the 1988-89 baseline year than students at 
the comparison school— a result that did reverse the finding for 1989-90 minus 1988-89. At 
the same time, no other improvements for demonstration students were recorded. To 
understand better what may have been happening at this site in 1990-91, that is, what the 
effects on students may have been of the changes in project implementation that occurred 
during that year, it is helpful to examine the results for Portland based on the 1990-91 
student cohort as well. By the end of 1990-91, Portland Cluster Plan students in the 1990-91 
student cohort had recorded fewer absences and higher grade averages relative to the 1989- 
90 baseline year than had the comparison students. These findings, together with the 
similarly positive 1990-91 finding on absences for the 1989-90 cohort, suggest that the 
formal and informal counseling provided by the new Cluster Plan staff at George Middle 
School in 1990-91 (to both the 1989-90 and 1990-91 student cohorts) may have borne fruit 
in terms of improving student attitudes and adjustments to school. The maintenance of 
improved attendance rates relative to comparison students in 1991-92 only underscores this 
sense of project accomplishment. Having said this, however, there remains the question of 
why grade averages for the 1989-90 demonstration students (i.e., the 1989-90 cohort 
treatment sample) failed to improve in 1990-91, a question we cannot answer. 

A second cohort was selected at both the Portland and Broward County sites late in 

1989- 90 because both seemed to promise positive outcomes for students based on the 
implementation study site visits. These selections were made (indeed, had to be made) before 
the 1989-90 data from these sites could be coded, edited, and analyzed. While in the case of 
the Portland project the selection may have helped to identify the positive effects of a change 
in implementation at the site, in Broward County the selection of a new cohort made it 
possible to fairiy assess the performance of the Model School Adjustment Program. Because 
an error had been made in identifying the comparison sample for the 1989-90 cohort, the 

1990- 91 cohort provided the only approximation of a matched-pairs design at the site for the 
evaluation. The result shown on Table 15 is not a strong endorsement for the program, 
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however. As related in the project description presented earlier in this chapter, neither 
analyses by year or by semester produced evidence of any additional positive outcomes that 
might be attributed to the MSAP. 

Similarly, the two positive accomplishments shown on Table 15 for the Tri-School 
project in San Antonio provide little consistent evidence of project success. Grade gains 
made early in the program were not sustained, and dropout rate differences at the Third 
Follow-Up were not evidenced at any other time-point or at the same time-point using a rate 
based only on school-classified dropouts. What is imponant here is what might have 
happened were the project to have been provided with consistent oversight throughout. At the 
same time, it is difficult to be certain that an approach favoring use of pull-out remediation 
could have sustained effects in this situation, and it is an open question as to whether the 
project's coordinated approach among the middle school and high school sites or the 
elimination of the pull-out approach in the third year actually accounted for the relatively 
lower dropout rate evidenced in 1991-92. 

Interpretation of Findings 

Reducing the stress for students associated with the middle school experience, which 
was noted at the beginning of this chapter, may be the greatest challenge for interventions 
aimed at preventing school dropout at this level. In particular, middle-school students who 
are at risk of academic failure at school may be more able and willing to invest energy and • 
time in learning at school if they feel valued or cared about by teachers and other school 
staff. From the perspective of the school, the extent of caring for students may be 
communicated subtly by the way in which the school experience is organized. It is 
communicated in more direct ways by the actions of the staff working with students. 

Organizational structure. The structures of the four demonstrations described in this 
chapter (excluding New York) may be divided roughly into two groups, although differences 
in Uie handling of service elements within projects and changes in orientation over time have 
occurred. First, the Broward County and San Antonio projects followed a pull-out strategy, 
which has students pulled from their regular classes to receive specialized service for one 
period per day. Second, the Portland and Denver-Lake projects sought to integrate services 
into the daily schedules of students. More specifically, in 1990-91 especially, Ponland 
counselors made themselves available to students informally on an as-needed basis before 
and after school, during school breaks, and during the lunch period. In Denver, the 
integration was most complete in 1989-90, when the school day for Gold Team students was 
entirely reorganized. 

Certainly it would be a mistake to assume that Broward County and San Antonio only 
offered pull-out services (e.g., both projects had counselors available to talk to students), yet 
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it is the case that during 1989-90 at both sites and during 1990-91 at Broward County 
students could expect to be pulled from class on a regularly scheduled daily basis to receive 
supplementary academic instruction. For the students involved, whether or not the project 
staff persons at these sites evidenced caring attitudes during the period of project contact, 
their regular day was unchanged. There is also the concern that pull-out programs, no matter 
how well designed and instructionally sound, convey to students the message that they are 
different and, perhaps, not as capable as other students. This stigma may negatively affect 
project accomplishments, causing students to be less interested in active participation during 
the pull-out period and more careful about demonstrating accomplishments developed during 
the period to students in other classes. 

At the Portland and Denver-Lake sites, any sense of stigma was removed by providing 
students either with a changed school experience or by creating a seamless extension to the 
regular program. In 1990-91, Portland counselors were always there for students, to listen 
and talk with them and to intercede with teachers when requested to do so, and the students 
responded by seeking them out during free moments. The relationships that were developed 
gave tangible support to focused project activities (e.g., discussion groups) and only served to 
help students with any problems they may have encountered during other school experiences. 
In Denver-Lake, particularly in 1989-90, the changed school day seemed to have resulted in 
a climate students perceived as more caring, fair, and safe. As much as anything, students on 
the other teams at Lake Middle School wished that they had been chosen for the Gold Team. 

Effective strategies. The efforts at Portland and Denver-Lake are noteworthy also in 
that very different strategies were used to provide a sense of caring and a sense of a nurturing 
environment for students. As noted, Portland relied on counselors who came to be perceived 
as friends whereas Denver-Lake relied on a family orientation; that is, a teacher team 
working with a student team on joint projects. In both cases, the structures of the programs 
enhanced the caring instincts of the school personnel, allowing them to be clearly 
communicated to students. 

As-needed informal counseling provided to middle school students throughout the day 
seems simple enough to offer, depending upon fiscal resources. However, it is easy to see 
that attempting to implement such a strategy with staff who are willing and able to listen to 
and talk with students can be problematic if the larger body of school staff and parents are 
not properly prepared. Teachers might regard the activities of the counselors as detracting 
from their relations with the students or see these counselors increasingly as meddling in 
issues that only concern them and their students. Parents might see counselors as providing 
unwarranted advice in areas outside of the school's additional concerns. In Portland, steps 
were taken to inform both teachers and parents about the aims and methods of the counseling 
staff. Individual and group meetings were held with teachers, and parent meetings at the 




school and home visits by the counselors were made. Strategies such as these should be 
included in any effort to adapt the Portland experience to other school settings. 

The strategies used by the Denver-Lake project to change the school climate for 
students at risk are likely to be more difficult to implement in a new school setting, since the 
preparatory activity to obtaining the required cooperation of and authority for teachers would 
seem more demanding. In addition, maintaining these strategies at a site so that they remain 
effective and fresh for teachers and students also requires effort. As evidenced in Denver- 
Lake in 1990-91, budget restrictions and changed priorities can alter the structure of 
programs drastically. In addition, although the project had envisioned keeping the same 
teacher team together with the same students throughout those students' tenure at Lake, not 
one of the five teachers who joined the team in 1988-89 was with the project in 1990-91. In 
Chapter 6, we address the topics of sustaining and replicating innovative strategies and 
discuss further some of the barriers and incentives foi school changes of the sort described in 
this chapter. 
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Chapter 5. Outcomes for High School Students 



High school is the critical time for retention of students. Upon enrolling in high school, 
some students may feel alienated in the larger and less personal environment and may never 
make the attachments needed to sustain their interest and attendance. Comparing the middle 
school and high school demonstrations participating in this evaluation during 1989-90, for 
example, the average rate of absence increased by about 2% from the middle grades to the 
high school years (i.e., from 5% to 7%, or about 13 days absent per school year). During high 
school, about 20% of the variance in absences during one year can be accounted for by 
absences during the previous year, indicating there is a definite pattern of recurring absences 
among students at risk at this level. High school students may also find themselves 
increasingly more likely to leave school befor<* graduating because of employment, marriage, 
pregnancy, or peer pressure. Indeed, even those SDDAP projects for high school students 
that included a paid-work component as an incentive often made special efforts to guard 
against students becoming overiy involved with their jobs at the expense of school 
attendance or enrollment. For many high school students, however, their break with school 
likely occurred years before, and it is only when they reach 16 years of age that dropout 
becomes a viable option. After leaving school, many students may regret their action, but, 
having made the break, they often find it difficult to return. 

The SDDAP projects that worked with high school and re-entry youth were the most 
likely to name dropout prevention as the pre-eminent aim of their work. In attempts to 
achieve this aim, the majority of projects at this level adopted comprehensive service 
strategies; that is, they offered academic, social support, counseling, and vocational/career 
services (in combination) to students either in alternative schools, schools-within-schools, or 
other adaptations of regular-school settings. In fact, the project components at this level 
offering comprehensive services were the most likely to be fully utilized. Eighty-five percent 
of the components offering comprehensive services to students in grades 9-12, for example, 
reported operating at or beyond 91% of capacity in 1990-91, and 57% of the project 
components serving re-entry students reported this level of utilization. By contrast, 
substantially fewer of the project components at these levels offering more limited ranges of 
services reported such high levels of utilization. For example, combining the data for 
components serving students in grades 9-12 and re-entry students, 74% providing 
comprehensive services reported operating at or beyond 91% of capacity as compared to only 
41% of those providing more limited, or focused, services. 

The seven projects participating in the in-depth evaluation that served high school and 
re-entry students used different strategies to meet varying student nwtivations for leaving 
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school prematurely. The projects were in Des Moines, Iowa; Memphis, Tennessee; Coleman, 
Texas; Aiken, South Carolina; Ethete, Wyoming; Denver, Colorado; and Carbondale, 
Illinois. Each of these projects is described in greater detail below, and data on student 
outcomes are presented for each project. All the seven projects participated in the evaluation 
since 1989-90, with the exception of the Denver-Discovery site, which began participation in 
1990-91. In Carbondale, an alternative to the matched-pairs evaluation design was used. 
Specifically, about 200 students identified by the high schools in the area as being at risk of 
dropping out were selected in 1989-90 and were followed up by the study through 1991-92. 
The actual rates of dropout from school, entry into Operation Rebound, and graduation or 
dropout from Rebound are presented in this chapter as the key outcomes of the project effon. 

For all the sites except Carbondale, survival analysis was used to estimate differences 
between project and comparison students in the time to (first) dropout from school. This 
analysis method assesses the holding power of the projects for individual students, and it 
provides a coniplementary measure to comparisons of annual dropout rates, which provide a 
series of discrete snapshots of students' status over time. 
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New Horizons/School-within-a-School 
Des Moines, Iowa 



Overview of Project Activities, 1989-90 

The New Horizons (NH) project provided counseling, attendance monitoring, career- 
related instruction, and work experience to high school students at risk throughout the Des 
Moines public school system since 1968. The School-within-a-School (SWS) program was 
separate and operated in five Des Moines high schools, providing smaller classes with self- 
paced, personalized instruction. SDDAP funding enabled the New Horizons project to extend 
its supervised work experience component to SWS students. The primary goal of the 
combined NH/SWS project for students in grades 9-12 was to improve students' school 
achievement and increase their graduation rates, specifically by offering work experience to 
help students focus on learning for a purpose, i.e., to make the connection between what they 
were being asked to learn in school and its application to the world of work. While the 
demonstration served students at all high school grade levels, particular emphasis was placed 
on the enrollment of 9th- and lOth-grade students. 

NH/SWS participants attended SWS classes during the school day, receiving academic 
instruction in math, English, and social studies, as well as instruction in life skills and career 
exploration activities (e.g., resume preparation). The SWS classes featured the regular 
curriculum but in classes of no more than 15 students, allowing greater individualization of 
instruction. As long as they remained enrolled in school, NH/SWS students were placed in 
after-school jobs for an average of 3 hours per day, or 15 hours per week. Three grant- funded 
work advisors held weekly group sessions for participants on job-related behaviors. These 
advisors also made the job placements and monitored students' performance through weekly 
telephone contacts with employers and monthly visits to job sites. They also visited each 
student's home every semester. In addition to these advisors, one counselor at each of the five 
high schools provided referral services and individual and group counseling to NH/SWS 
participants. 

Private-sector employers were responsible for paying the students they employed and 
for naming an on-site work supervisor to interact with the school-based work advisors. The 
wages paid to students varied, and students employed in the public sector (e.g., hospitals and 
schools) were paid by New Horizons, with most receiving $4.20 per hour. Most jobs 
appeared to involve routine tasks, such as sorting mail, stocking supermarket shelves, and 
sweeping floors. 

Project staff. Three work advisors shared responsibility for the five schools, with two 
working half time at two schools each and the remaining one working half time at one 
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school. All had at least 20 years of experience as teachers, but none had ever served as a 
counselor. While they were responsible for placing students in jobs, they were not 
responsible for recruiting prospective employers. Rather, they used lists of employers 
generated by other New Horizons staff. The directors of NH and SWS were located in the 
same building, an annex of the Des Moines Public Schools Administration Building, with the 
NH director serving as project director of the collaborative project. 

Participant selection. Approximately 240 students were in the SWS program at the five 
participating schools. All were eligible for job placement services available through the joint 
program. The vast majority of students were placed in jobs; however, a small minority were 
not deemed mature enough to be placed in jobs. Nevertheless, these students were included 
in the weekly group sessions to prepare them for future placement. Stp.dents were referred to 
the SWS program by their junior high counselors or teachers based on their grades, ITBS 
scores, and attendance. Approximately 80% of the students who received NH/SWS services 
were white. 

Project services. Program students received an average of 10 hours of service per week 
for 22 weeks. The SWS program provided transitional academic instruction in math, English, 
and social studies for youth at risk in their freshman and sophomore years, as well as special 
academic assistance to students in their junior and senior years. Classes were taught by 
regular teachers who shared time teaching regular classes and SWS classes. The SWS 
curriculum was identical to the regular curriculum but was offered in classes no larger than 
15 students. Average class size was actually about 12 students. Because of the small class 
sizes, teachers were more able to individualize the instruction. 

In addition to the work advisors, each SWS site had its own counselor as well. 
Counselors provided referral services, individual counseling largely on a drop-in basis, and 
some group counseling. The student-counselor ratio in each school was approximately 60 to 
1, with the exception of East High School, where the counselor also acted as counselor for 
special education students. 

Besides the work experience component, work advisors at each school held weekly 
group sessions for participating students on such topics as on-the-job behavior, resume 
writing, interview skills, and more. Work advisors also visited each student's home once 
every semester. 

Students expressed the greatest satisfaction with the personal attention and assistance . 
they received from their SWS teachers and from their NH/SWS work advisors. They were 
not as enthusiastic about their actual work experience, some complaining that most of their 
work was boring. Only two students dissented from this view, with one commenting that "I 
am more responsible at home since I started working." Another added that "I used to skip a 
lot in junior high, but not anymore. I want to work...The program is great." 
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Notable Changes in Opera'dons During 1990-91 

Basic operations did not change during 1990-91. Emphasis continued to be placed on 
improving attendance rates and reducing dropout rates. Supportive service plans were 
developed for each student; home visits and personal contacts with parents were made; the 
ratings of students' job supervisors were collected by NH/SWS staff to measure work 
performance and attitudes toward work. An average of 44 career-related sessions were held 
at each of the five SWS schools during 1990-91. 

Status in 1991-92 

The activities of the New Horizons Program continued in 1991-92 without appreciable 
change; local funds were guaranteed for at least the 1992-93 school year, and additional 
grant applications were expected to be submitted during that time. Administrative support 
was particularly important in this district with site-based management, and the project 
director attributed the success of New Horizons in gaining support to the success the program 
achieved in working with difficult student populations. He believed that the program had a 
significant impact on the district, creating more positive attitudes toward technology, 
involving families and other members of the community, and demonstrating the value of 
work experience to motivate students, to demonstrate the inter-dependence of workers, to 
increase students' self-esteem, and to give them a sense of personal power. 

Project Outcomes 

Since the combined NH/SWS program was operating in five Des Moines high schools, 
we selected comparison students at each of these schools as well. Specifically, we selected 
1 1 1 students at random from among the total number of NH/SWS participants at these high 
schools, and our comparison sample was comprised of 98 students at risk, mostly 9th and 
10th graders, who participated in (either) the NH program (four students), the SWS program 
without the supervised work experience component (nine students), or who were enrolled in 
the general school programs at the sites (85 students). Selecting students as comparisons 
from the same schools at which the NH/SWS program was operating raised questions of 
possible selection bias; that is, that the students in the program might have been judged as 
more capable despite various at-risk conditions or that they were more motivated to seek 
special services. We found no evidence of the former being the case (i.e., interviews failed to 
disclose any systematic efforts on the parts of teachers or counselors to find the best students 
for the program), and the lack of specific entrance requirements (e.g., application forms) 
suggested that NH/SWS students who were identified for the program were assigned without 
much opportunity or necessity for their volunteering for services. It is the case, however, that 
the compai-ison group at this site was comprised of a greater number of over-age 9th-graders 
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than were in the treatment group, and that the treatment sample had a greater number of over- 
age lOth-graders and 1 1th- and 12th-graders generally than did the comparison sample. 

Baseline differences existed between the program and comparison samples in terms of 
the grade averages that had been earned during 1988-89, the year prior to formal entry into 
the program. Based on records data available for 99 and 81 students, respectively, we found 
that NH/SWS students had recorded higher grade averages (i.e., 1.67 versus 1.28) during the 
baseline year. In contrast, there were no baseline differences between the groups in percent 
days absent; the rates for NH/SWS students and comparison students were 5.9% and 6.5%, 
respectively, a difference that was not statistically significant. 

The more intensive follow-up of students at the Des Moines site, which involved the 
gathering of grade and absence data from school records, was discontinued after 1989-90. 
This was due to our judgment that the NH/SWS "treatments" being delivered to students at 
each of the five high schools were sufficiently different to make specific statements about 
"NH/SWS strategies" impossible. Notwithstanding these treatment differences, however, 
students in Des Moines were continuously followed through the 1991-92 school year with 
respect to their enrollment status. 

Improved academic performance and improved attendance were judged by the 
implementation study team to be plausible short-term outcomes of project efforts in 1989-90. 
The key longer-term outcome that was identified for the project was increased student 
persistence to graduation. More specifically, these outcomes were deemed plausible 
because project staff were judged to have carefully integrated a key incentive, i.e., paid work, 
with performance expectations regarding both performance in remedial classes and school 
attendance. 

Student academic performance. In 1989-90, some evidence of improvement in grade 
averages was found for the treatment students in Des Moines (i.e., they raised their averages 
by about .147, or about one-fifth of the baseline mean standard deviation, while comparison 
students lowered their averages by about .189); however the response rates on which this 
result was based were low (about 59%). When adjustment was made for nonresponse and the 
analysis was re-run, this gain was not significantly different from that recorded by the 
comparison group, although the direction of the effect continued to favor the treatment 
group. Given the higher grade averages of NH/SWS students during the baseline, it also is 
difficult to attribute this gain to the content of the project alone; that is, the selection 
mechanism used to assign students to the combined program may have served to involve 
students in the project who were more likely to take advantage of project activities. In 
contrast to the data from school records, student self-report data related to perceived 
improvements in class participation and homework (which were collected in Spring 1990) 
did not evidence any differences between the groups. 
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Student attendance. No statistically significant differences were found between 
NH/SWS students and the comparison students in changes in absence rates from the baseline 
to the first follow-up year. Looking at each group, however, NH/SWS students increased 
their rates of absence by about .5% while comparison students increased their rates of 
absence by about 1.8% (or almost one-third;!of the baseline standard deviation for the group). 
The rates of data availability in this case were 86% and 65%, r<^spectively, for the two 
student samples. As in the case of academic performance, no differences between the groups 
were revealed by 1989-90 student self-report data on perceived improvements in getting to 
class on time. 

Student persistence to graduation. With some evidence of having raised students' 
grade averages and having helped to contain students' absence rates (although the differences 
between the treatment and control groups were not statistically significant on this latter 
measure), it was perhaps less surprising that the combined NH/SWS program succeeded in 
retaining students. Based on virtually complete data for the student samples, dropout rates for 
both 1989-90 and 1990-91, whether computed based on school-classified dropouts only or 
on school-classified dropouts plus moved and expelled students, were consistently lower for 
NH/SWS students, and the differences in every case between NH/SWS students and the 
comparison students were statistically significant at p<.05. For 1991-92 (also based on 
virtually complete data for the student samples), dropout rates for NH/SWS students, when 
moved and expelled students are considered along with school-classified dropouts, also were 
statistically significantly lower than those for the comparison sample. The difference in rates 
between the groups for 1991-92 when school-classified dropouts only were considered also 
very nearly approached statistical significance. In this case, the dropout rate for NH/SWS 
students (in 1991-92) was 17%, compared to a 30% rate for the control students, and the 
level of significance for this difference was p<.059. 

In addition to comparison of dropout rates, survival analyses using the Cox Proportional 
Hazard test to compare the rates of persistence of the student samples in each of the three 
years from 1989-90 to 1991-92 showed that NH/SWS students remained in school longer. 
The results of these analyses were the same when either school-classified dropouts only or 
dropouts, moved, and expelled students were considered, and they were the sanw whether or 
not the analyses were stratified on the basis of gender, race/ethnicity, and age differential 
with respect to grade. The results of survival analyses for the 1989-91 period overall 
confirmed the findings for the individual years; that is, NH/SWS students remained in school 
longer. In addition, these analyses suggested that the risk factor associated with dropping out 
for NH/SWS students was not constant over time but increased somewhat from year to year. 
This pattern may indicate the changing compositions of the treatment and comparison groups 
over time, or it may reflect some change in the salience or lasting-effect of the treatment for 
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participants. In any case, these results do suggest the NH/SWS approach had definite, 
positive effects on students' persistence to graduation. 

Factors affecting realization of outcomes. The key factors that affected dropout 
prevention outcomes in Des Moines appear to have been (1) the integration of paid-work 
experience, as an incentive, into an established school-within-a-school program, and (2) the 
articulation of specific numerical goals for reducing the dropout rates of high school-aged 
students at risk of failure. In addition, the fact that the work experience component of the 
program was well-established in the community meant that (1) work experience coordinators 
had models of behavior in carrying out their responsibilities tc^sure that students' jobs did 
not interfere or jeopardize their schooling, and (2) the majority of employers relied on by the 
program understood clearly that the work experience of students was intended as a 
complement to their school work. Finally, this demonstration had an experienced 
management team familiar with both the NH and SWS programs and their staffs, thereby 
ensuring a sensitive approach was taken in combining these strategies so that staff from 
neither of the programs felt left out or that their missions had been unduly compromised. 
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Memphis Partners Collaborative 
Memphis, Tennessee 

Overview of Project Activities, 1989-90 

The Memphis Partners Collaborative (MPC) was a Saturday program for lOth-graders, 
which was offered through the joint efforts of the Memphis City Schools (the SDDAP 
grantee), Memphis Partners, Inc. (a community-based, not-for-profit group), the Ford 
Foundation, and other cooperating organizations. Project activities were held for six hours 
every Saturday for 17 weeks at several local college campuses, including Memphis State 
University, Shelby State Community College, and State Technical Institute. 

Approximately 40% of an average project day was devoted to academic enrichment or 
remediation in mathematics, reading, and test-taking strategies; another 40% of the day 
focused on job readiness and employability skills (e.g., how to prepare for a job interview); 
the remaining 20% of the time was designed to (1) increase motivation, self-esteem, stress 
management, and problem-solving skills, and (2) provide health and drug-related counseling. 
Transportation, lunch, and various daily rewards to enhance participation (e.g., gift 
certificates for good attendance records) were provided. Students who completed the 
program successfully were placed in summer jobs by Memphis Partners, Inc., which 
monitored the students' work and offered additional training and assistance as needed. Some 
of these jobs could be continued on a part-time basis into the school year if students were 
amenable and work monitors, or counselors, approved. 

Project staff. Teachers involved in the project were career ladder teachers who 
volunteered for involvement and received extended contracts (i.e., additional pay). The 
project also employed counselors/job developers whose responsibility was to identify jobs, 
arrange interviews for participants, and monitor participants' performance on the job. The 
project director was located in the central office of the Memphis Public Schools. She 
oversaw all projects in the collaborative, identifying and hiring staff, ensuring availability of 
resources, and monitoring project implementation. 

Participant selection. Students were identified for participation primarily by their home 
school principal, counselors, or teachers. Selection criteria included being over-age for grade, 
excessive absences, high mobility, JTPA eligibility, and academic deficiencies. Should they 
meet all the criteria, students were selected on a first-come fu^t-served basis, based on the 
date of receipt of their completed applications. After acceptance, students and their parents or 
guardians had to attend an orientation meeting where they signed Student/Parent Agreements 
promising that the student would maintain good work habits on the job and would strive to 
improve grades, attendance, conduct, and punctuality at (regular) school. Students had to 
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maintain good attendance at the Saturday program and were dropped if they had three or 
more unexcused absences. According to local tabulations, the project screened approximately 
350 youth and served around 237 over the course of a year. Approximately 187 participants 
(or 80% of those selected) were estimated to have completed the project in any one year. 
Very nearly all MPC participants were Black. 

Project services. Memphis Partners participants attended six-hour sessions each 
Saturday for 17 weeks. The project provided career counseling (which included development 
of career plans and tracking participants' development in relation to these plans); incentives 
to encourage attendance and participation, with various incentives (such as food, tapes, and 
trips) provided by business and industry; job placement (including monitoring) — full-time in 
summer and part-time after school; mentoring; cultural/social awareness activities; and 
volunteerism (students were required to volunteer for 30 hours per year). In addition, the 
parental involvement component included an orientation for parents, home visits, periodic 
meetings, and written progress notes. 

According to respondents, the key components of the program were the self-esteem- 
building activities and the employment-related aspects (including career awareness, 
employability skills, and the jobs themselves). In order to simulate the workplace, the project 
implemented strict behavioral, attendance, and dress-code rules. Students were taught to 
understand the need for these types of rules and the importance of "internalizing" them, 
which was a key objective of the project: to teach youth appropriate workplace behaviors and 
the skills to seek, obtain, and retain jobs. A corollary objective was to make youth realize the 
importance of a high school diploma in terms of earnings and employment stability, as well 
as to enable them to continue their education and training after high school. 

Notable Changes in Operations During 1990-91 

Seven high schools were targeted for participation in MPC in 1989-90; hov/ever, staff 
believed that discipline problems and cliques from rival high schools negatively impacted the 
program during that year, so, in 1990-91, the service area was expanded. In 1993-91, only 
about one third (34.5%) of the participants were from the seven 1989-90 target high schools; 
the remaining participants were from 16 other district high schools and one high school in 
neighboring Shelby County. In addition to expanding the service area, efforts w jre made to 
include some students with higher academic abilities, to provide a richer mix fo.- peer work 
and academic enrichment. A local evaluator commented that, in her opinion, much of the 
service expansion was motivated by a desire to streamline program administration, although 
this sort of expansion would seem to introduce greater complexity in program operations. 

To make the Saturdays less like school days, more speakers and more vari;d activities 
were provided than during 1989-90. Special presentations were provided to bu Id self-esteem 
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and ethnic pride among enrolled students, most of whom were black; e.g., Nubian dancers 
performed; a local judge and a local newscaster, both black, addressed the group. An 
Employer Day was held, with a seven-member panel talking about what they look for in 
employees and answering questions from students. The same local evaluator who viewed 
expansion as an administrative move regarded these sorts of large group sessions as more 
organized when special presentations were featured. 

The two counselors from Memphis Partners maintained close contact with the students' 
schools, visiting schools once a week to foster support from administrators and staff, talking 
with students informally, checking on attendance, and ensuring that students were following 
through on job interviews and other jol>-related tasks. The counselors also made frequent 
calls to parents— an estimated 50 per week, most of them concerning project activities and 
potential jobs for students rather than problems in which the students were involved. 

Status in 1991-92 

The Memphis SDDAP program provided an umbrella for services for at-risk students, 
and that coordination continued in 1991-92, although the number of services was reduced. 
The Memphis Partners program, which was the focus of the in-depth evaluation, continued to 
operate during the 1991-92 school year with support from JTPA and local businesses, but 
served fewer students. The SDDAP project director returned to a teaching assignment in one 
of the district's schools. Because of local economic conditions, community support for the 
program was becoming more difficult to secure, and the longer-term prospects for continuing 
the Saturday program were uncertain. 

Project Outcomes 

Student cohorts were selected in Memphis in both 1989-90 and 1990-91, and 
appropriate baseline and follow-up data were collected from the program and comparison 
students in each cohort. Since the program required students to apply, we selected our 1989- 
90 comparison group from students who were identified by project staff as also being at risk 
and who attended Memphis high schools that were not eligible for MPC participation during 
that year. In 1990-91, when the target population for the program was expanded, we selected 
comparison students from among those young men and women who reportedly had 
submitted completed MPC applications (and who had satisfied all entrance criteria) but who 
could not be served (i.e., their applications arrived somewhat later and, by then, all available 
placements in the program had been filled). 

The sizes of the student samples in the 1989-90 student cohort were 97 and 78 students, 
respectively, for the Memphis Partners and comparison groups. In 1990-91, 100 Memphis 
Partners participants and 98 comparison students were randomly selected for the study. The 
rates of records data availability and student responses to the survey instrument in 1989-90 
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were approximately 80% and 66%, respectively, for the treatment students. These rates for 
the comparison students in 1989-90, however, were substantially below 50% (e.g. 37% for 
data on grade averages and only 24% for student self-evaluations of performance). Only in 
the case of records data on enrollment status over the years 1989-90 through 1991-92 were 
data availability rates for the comparison students selected in 1989-90 above this minimally 
acceptable level; in these cases, the availability rates were about 75%. For this reason, results 
of comparisons based on the 1989-90 student cohort in Memphis must be interpreted with 
the greatest caution. In contrast, the response rates and rates of data availability for the 
1990-91 student samples (i.e., the 1990-91 student cohort) were greater than 50% for all 
measures and for all (comparative) analyses, with the majority of analyses during the 
1989-90 baseline year based on 100% and 75% of the data for the samples, respectively. 

Considering the baseline records data for both student cohorts and the pre-treatment 
reports for the 1990-91 cohort made in response to the first administration of the survey form 
in Fall 1990, there is some evidence of differences between the MPC and comparison 
samples. For both cohorts, for example, the grade averages earned by MPC participants 
during the baseline periods were higher than those earned by the comparison students. In 
both cases, MPC participant averages were just about one-half a grade-point higher. 
Similarly, although the absence rates between the samples in both cohons were not different 
statistically speaking, in both cases the rates for the MPC participants appear to be less by 
about 1.5% to 2%. Finally, looking only at the survey responses for the 1990-91 cohort, 
several self-evaluation items (e.g., items related to doing homework and working on class 
projects) and reports of school-related attitudes (e.g., the perceived importance of graduation 
and the certainty of graduation) tend to suggest MPC participants were more confident of 
their abilities and of the likely results of their efforts. Given that a selection process was 
involved in admitting students to the MPC, we worried that, despite best efforts, our study 
samples might be sufficiently different to make interpretation of results difficult. 

To begin to understand the reasons why students in Memphis were choosing to apply 
(and to apply in timely fashion) to MPC, we worked with school officials and helped them 
carry out a special survey of the program and comparison students. This survey (which was 
modeled after the one used with the Broward County project and described in the previous 
chapter) consisted of seven items asking students to indicate by ratings the sorts of factors 
that might have nwtivated them to apply or caused them not to apply for MPC. 

In general, students in both groups acknowledged that MPC (or a program like MPC in 
the case of the comparison students) could help students get jobs and do better in school, 
although there was a modest difference between them as to whether MPC would help them 
learn useful skills (i.e., students in the comparison group were somewhat less certain of this 
outcome). The major factors distinguishing the groups' motivation to participate seemed to 
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involve students' finding the time for the program and the relative amounts of support 
students expected from parents and friends. More specifically, 73% of MPC participants said 
they had no trouble finding the time on Saturdays to participate, as compared to only 47% of 
the comparison students. Similarly, while 85% of MPC participants reported they were given 
a lot of support by their parents in their decision to enroll in the program, only 72% of the 
comparison students reported they would likely receive support from Uieir parents in making 
such a decision. Interestingly, MPC students were less likely to report they received a lot of 
support from their friends for their decision to enroll than were the comparison students (i.e., 
22% versus 41%, respectively). While the results are suggestive, their interpretation is 
complicated by the fact that the two groups were responding to different situations— one real 
and one hypothetical. Nevertheless, taken together with the baseline and pre-treatment survey 
data described above, these suggested differences between the 1990-91 student samples in 
finding time available for the program and getting parental support for participation 
underscore that the samples hardly appear to be matched and that interpretations of 
comparative results must be drawn with caution. 

Areas in which possible short-term outcomes seemed likely for MPC in 1989-90 
included employment and improved self-esteem Possible future outcomes at that time were 
judged to include improved attendance, improved academic performance, and increased 
persistence to graduation. In 1990-91, we were more inclined to regard improved 
attendance at school as a possible short-term goal, since MPC counselors regularly visited 
students' home schools and monitored their attendance. (This demonstration of counselor 
interest and follow-up may have been one reason that MPC participants reported more 
frequent contacts with counselors than did comparison students in 1990-91.) In addition to 
this sort of monitoring activity, rewards on Saturdays were sometimes given to students who 
recorded few or no absences during the previous week, and counselors did check with 
students on Saturdays regarding why they were absent from school the previous week. The 
monitoring of grades was less systematic, and the quality of the small group instructional 
sessions provided to students on Saturdays was, in our opinion, much less than would be 
desired. Too often, teachers in the Saturday classes used chalk-and-talk and drill-and-practice 
approaches commonly used with large classroom groups, when the smaller numbers of 
students in these classes could have prompted the use of more creative instructional strategies 
(e.g., individualized instruction or cooperative learning activities). 

Student employment. The student survey did collect data on students' employment and 
earnings; however, the data collected from the 1989-90 student samples were without 
baseline measures of employment status, and the data collected from the 1990-91 student 
samples were gathered so soon after "graduation" from MPC that it is difficult to say whether 
students participating in the program had already begun their work assignments. Reports 
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prepared by the Research Services Division of the Memphis City Schools do contain 
information on MPC job placements, and based on these data it is evident that students who 
were 16 years of age or older, who participated in MPC and completed the program, were 
likely to get jobs. 

For example, in 1989-90, the Research Services Division reported that 182 of 237 MPC 
participants completed the 17-week session (i.e., they "graduated"). Of these 182, 143 (or 
79%) were employed in entry-level jobs. Of the remaining 39 students, 25 were younger than 
16 years of age and were placed as volunteers, and 14 were not placed. In 1990-91, 208 of 
the 252 MPC participants completed the 17-week session, and of these 208, 154 (or 74%) 
were employed— more than a third by McDonald's or Liberty land, a local amusement/theme 
park. Thirty-seven of the 54 students who were not employed were reported as being under- 
age. For employed students in 1990-91, the Research Services Division reported that about 
67% received the minimum wage of $4.25/hour. 

Student self-esteem. In 1989-90, surveys were administered to students only once, in 
the spring of the year. Because of problems in obtaining the cooperation of comparison 
students who were selected for the 1989-90 student cohort for the evaluation, an insufficient 
number of returned instruments was available for extensive analysis, as was noted earlier. 
Keeping in mind our concerns about the poor response rate among comparison students, we 
did compare the self-esteem scores of the 67 responding MPC participants and the 19 
responding comparison students, response rates of 72% and 32%, respectively. Based on 
these unadjusted data, MPC participants did evidence higher scores on the Rosenberg Self- 
Esteem Scale than did the control students. However, the small and unrepresentative sample 
sizes together with the lack of baseline scores on the Rosenberg scale make these results 
questionable at best. 

For the 1990-91 student cohort, students' self-esteem was measured three times: at the 
beginning of the 1990-91 school year, at the end of that year, and at the end of the 1991-92 
school year. Acceptable response rates were obtained from each of these administrations 
(e.g., response rates of 91% and 75%, respectively, for the treatment and comparison samples 
at the time of the second-follow-up). Comparisons of the extent of change in self-esteem for 
MPC participants and comparison students evidenced no statistically significant differences 
from the baseline over either of the periods; in fact, for both groups self-esteem scores 
decreased from the baseline by the second-follow-up assessment in 1991-92. 

Student attendance. Data from both the 1989-90 and 1990-91 student cohorts show 
that MPC participants recorded higher absence rates during the years in which they attended 
MPC Saturday sessions and in the years immediately following their participation in MPC. 
For both cohorts, however, comparison student absences increased at even higher rates over 
these periods. For the 1990-91 student cohort, fo^example, the MPC absence rate increased 
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by about 2.5% over the baseline year during their participation in the Saturday program and 
by 4% during the following year. For the comparison students selected for this cohort, the 
absence rate increased by 6.5% over the baseline during 1990-91 and remained at this higher 
level through 1991-92. As a result, MPC students in the 1990-91 student cohort had 
statistically significantly fewer absences relative to their peers (and relative to the baseline) 
by the end of the first-follow-up period (i.e., 1990-91). MPC students in the 1989-90 student 
cohort had relatively fewer absences (i.e., relatively lower gains in absences measured in 
terms of the comparison group and the baseline) by the end of both the first- and second- 
follow-up periods. As noted previously, however, the rates of records availability for 
comparison students in these cases were low, necessitating caution in interpretation of these 
results. (In fact, during the second-follow-up, records data could only have been collected for 
66 comparison students for the 1989-90 cohort, as 12 of these students dropped out in 1989- 
90; for this reason, while records data were obtained for only 29 comparison students during 
both the first- and second-follow-up, the rate of data availability for the students during 
1990-91 was 7% higher.) 

The suggestion of some positive effect on attendance of MPC participation may be due 
to MPC attendance monitoring and reward activities; however, it is difficult to rule out the 
possible additional effect-of the MPC student selection process. Since MPC participants in 
both student cohorts appeared to have recorded somewhat lower absence rates during the 
baseline year than those for the comparison students, it may be that MPC strategies work best 
with students who are less likely to be chronic truants. If the applications process were 
modified, for example, or if applicants were equally eligible of receiving services regardless 
of the timing of their applications, it is possible that MPC program strategies would appear to 
have less effect on students' absence rates. 

In contrast to the data from school records, self-report data collected from the 1990-91 
student cohort at three time-points indicated that MPC participants were no more likely at 
either follow-up to feel themselves doing better relative to the baseline in getting to class on 
time than were the comparison students. 

Student academic performance. Neither the data from school records for either cohort 
nor the self -report data for the 1990-91 student cohort evidence any statistically significant 
differences in academic performance gains between MPC participants and the comparison 
students. For example, grade averages for the 1989-90 cohort showed MPC participant 
averages were about one-quarter of a grade point lower than the baseline year at both the 
first- and sccond-follow-ups, as compared to control student averages that ranged from about 
one-thiid lower to one-tenth higher for these years. For the 1990-91 cohort, MPC participant 
and control student averages decUned during both the follow-up periods (i.e., from about 
one-tenUi to one-half of a grade point for MPC participants and from about one-quarter to 
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one-third of a grade point for comparison students). In addition, MPC panicipants in the 
1990-91 cohort were no more likely than the comparison students to report themselves doing 
better relative to the baseline on class projects or homework by the time of any of the follow- 
ups. 

Student persistence to graduation. Overall, no differences were found in the dropout 
rates at the time of the first- and second-follow-ups for the treatment and control samples in 
either the 1989-90 or 1990-91 student cohorts. This is true both for rates calculated using 
school-classified dropouts only and for rates that combine with school-classified dropouts 
those students who had moved without confirmation of re-enrollment or who were expelled. 
The overall comparisons of these rates control for race/ethnicity, gender, and age with 
respect to grade; however, in the case of the MPC, differences in the patterns of dropout rates 
for particular groups of students are important because they suggest the sorts of students for 
whom the MPC may be more or less effective. 

For the 1989-90 student cohort, dropout rates for MPC participants ranged from 6.5% 
in 1989-90 to 10.8% in 1990-91 for school-classified dropouts and school-classified and 
probable dropouts combined. Comparison student dropout rates for the same periods ranged 
from 10.2% to 28.8% for dropouts only and from 20.3% to 30.5% for dropouts plus moved 
and expelled students. These percentages appear to indicate large differences between the 
samples, but statistically they are not significantiy different when the sample sizes are taken 
into account and controls for race/ethnicity, gender, and age differential are applied. This is 
because the statistical method being employed first calculates the extent of treatment and 
control group differences within each of the subgroups of students defined by the various 
combinations of race/ethnicity, gender, and age-differential designations and then aggregates 
the results across these subgroups. (Note that any subgroup in which both treatment and 
control group students are not represented is excluded from the aggregation.) 

Of the possible eight subgroups of Memphis students in the 1989-90 cohort, both 
treatment ^nd control students arc present in only four subgroups (this is because white 
students were included among the comparison students but not among MPC participants). Of 
the four subgroups with both treatment and control students, significant or nearly Significant 
differences are found in (only) two subgroups. Specifically, by June 1991, students who 
participated in the MPC in 1989-90 and were over-age, black, and female had dropped out of 
school at a rate of 30% as compared to a 0% dropout rate for control students with these 
characteristics (based on sample sizes of 20 and 8 students, respectively). In contrast, by the 
same date, students who participated in the MPC in 1989-90 and were over-age, black, and 
male (i.e., 15 students) had dropped out of school at a much lower rate than the 12 
comparison students with these characteristics. The dropout rate for MPC students in this 
case was just over 6%; for comparison students who were over-age, black, and male, the 
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dropout rate by June 1991 was 58%. These results are the same whether school-classified 
dropouts only or a more inclusive definition of dropout is used. Aggregating these results 
leads to findings of no difference overall between the MPC and control group students. 
However, it is important to note that the MPC experience in 1989-90 evidently contributed 
to different outcomes for male and female students who were black and over-age for their 
grades. 

Comparing dropout rates for the over-age, black females in the 1989-90 student cohort 
for the evaluation at the end of the second- and third-follow-up periods (i.e., 1990-91 and 
1991-92), we find that the differences in rates between MPC participants and comparison 
students in this category grow smaller— namely, a greater number of over-age, black females 
in the comparison group have dropped out by 1991-92 as compared to 1990-91. Because the 
patterns of dropout rates for the MPC students and comparisons who were over-age, black, 
and male remain the same for both these periods (i.e., many more comparison students than 
MPC participants drop out), the aggregated dropout rates for the site show significant 
differences between the two samples. Thus by 1991-92 for the 1989-90 student cohort, 
fewer MPC participants are found to have dropped out of school than comparison students. 
By 1991-92, the respective dropout-only rates for these groups are 21.5% for MPC 
participants and 59.3% for the comparison students; the respective dropout rates inclusive of 
moved and expelled students are 21.5% and 61%. 

The data for the 1990-91 student cohort do not reveal similar findings of significant 
differential effects for over-age, black males and females served by the MPC. The dropout 
rates for both these groups do, however, continue to be much higher than the corresponding 
rates for black males and females who were not over-age for their grades (e.g., 17.6% for 
over-age, black feinale MPC participants versus 2.2% for black female MPC participants 
who were not over-age). What these results for the two cohorts suggest is that this sort of 
program operating in Memphis must take special care to meet the needs of over-age students 
and must be especially concerned about the match between program offerings and the needs 
of over-age black students who are female. 

The results from survival analyses of these data are inconclusive, although a significant 
difference in the patterns of persistence for the MPC and comparison students in the 1989-90 
student cohort is noted during the 1991-92 school year. In that year, which would have been 
the 12th-grade year for both groups, MPC participants more than comparison students tended 
to remain in school for longer periods. 

Other findings. Relative to the comparison students and to their baseline reports, MPC 
students from the 1990-91 cohort reported they had more frequently talked with counselor(s) 
at school about their h.^.i school program during 1990-91. Similarly, by the end of the 
second-year follow-up (i.e., by June 1992), MPC students evidenced more ambitious plans 
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for school completion (i.e., how far in school they believed they would go) than did 
comparison students relative to the baseline year. Both these results might be attributed to 
MPC participation, as MPC counselors did make special efforts to talk with students about 
their programs in 1990-91 and an aim of the program was to raise the self-esteem and levels 
of ambition of students. To strengthen the case for attributing this latter result to the program, 
however, more of a consistent pattern of accomplishments in this area would be needed. 

Factors affecting realization of outcomes. The cooperative service arrangement 
involving the Memphis public schools and the community-based Memphis Pai;tners 
organization provided student-participants with job opportunities and counselors who 
actively monitored their attendance and performance in school and on the job. An apparent 
lack of effective communication between these entities, however, resulted in a less-than- 
effective instructional component for the Saturday MPC program. In addition, attribution of 
positive student outcomes to the contents of the MPC program is made difficult because the 
selection mechanism used by the program seemed designed to involve highly motivated 
students who had already demonstrated their concern for doing well in school. 

Why Some Students Left the Project or Left School 

Exit interviews were carried out with 25 students from the 1990-91 student cohort: 19 
students who dropped out of MPC, two students who dropped out of school without ever 
entering MPC, and four students who left MPC and school simultaneously. These interviews 
allowed students to talk about why they left (i.e., to give one or more reasons for leaving), 
who if anyone they had spoken to before leaving, changes they would recommend for the 
project or for their school, and what their current educational plans are. 

Students who left MPC reported that the time schedule was too demanding (six 
students) — in particular, the program began too early on Saturday mornings. Others of these 
students said that they left because they needed to work (three students), to care for a family 
member or handle a family crisis (three students), or because they had recorded too many 
absences or suspensions from project activities (three students). Ten students reported they 
had talked to no one about their decision to leave MPC; six students reported talking to 
project staff persons. Ten students also said they had no recommended changes for the 
project, whereas five students recommended the project begin later on Saturday mornings 
and have shorter hours. Other suggestions for program improvements included better 
teachers, holding sessions at the same site for the duration of the program, and providing 
child care. All these former MPC students reported they intended to remain enrolled in 
school and earn diplomas. 

MPC students who dropped from the project and from school did so because of poor 
attendance (two students), family relocation (one student), or the need to work (one student). 
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Most of these students consulted no one about dropping out and had little to suggest in the 
area of project improvements. All but one of these students reported plans to earn a diploma 
or a GED at some future time. 
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Cooperative Alternative Program (CAP) 
Coleman, Texas 



Overview of Project Activities, 1989-90 

Cooperative fiscal arrangements between school districts are unusual in Texas, and the 
Cooperative Alternative Program (CAP), as conceived and implemented, was aimed at 
demonstrating the willingness of seven school districts to create a unique governance and 
fiscal structure to respond to the needs of students at risk. The superintendents of the seven 
districts, with the leadership of the CAP principal, made up the management and governance 
board for the program, which was supported by SDDAP, state, and local monies, including 
A.D.A., vocational, compensatory, and special education funds. 

The CAP High School offered an alternative for students in the seven cooperating 
districts in the rural hill country of west central Texas. Its primary purpose was to provide 
remediation in basic skills and job-specific vocational training; it featured small classes, 
individualized instruction, individual and group counseling, and a student assistance program 
of tutorials. Except for science and vocational courses, students worked in individual study 
groups and could participate in accelerated programs if they wished. Students took basic core 
classes to gain the English and mathematics skills needed to pass the state competency 
examination and to graduate; they also took vocational classes in classrooms and labs and, in 
1989-90, were able to work for pay in the cafeteria and print shop on campus. 

The teaching and counseling staff at CAP was comprised of experienced teachers from 
among the seven participating districts and local community college who wanted to be a part 
of the program after visiting the school and classes and talking to other staff. Staff were 
encouraged to attend workshops and to observe in other programs in the area that served 
students at risk. In addition to the academic, vocational, and personal counseling and 
guidance services provided by the CAP school counselor, the entire staff informally assisted 
the students in finding solutions to many personal problems (e.g., obtaining food stamps). 

Project staff. The CAP school was staffed with one counselor, eight academic and 
vocational teachers, one resource teacher, and five aides. All the staff knew before they were 
hired that the students were at risk, and that many had major problems. Therefore, once 
teachers who were experienced and wanted to work with this difficult population were 
selected, project administration encouraged and supported staff to attend workshops, observe 
other school and community-based programs, and receive training in substance-abuse 
counseling and in other related areas. All the teachers were certificated, with several holding 
masters degrees and specialized certificates in vocational majors or special education. The 
counselor was a candidate for a professional counseling license and for a doctorate in 
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counseling. The principal in 1989-90 was in the process of completing a doctorate in 
administration. 

Participant selection. The primary screening device for the program was the Texas 
Education Agency's list of at-risk dimensions. Many of the CAP students included in the 
pool for this evaluation, who ranged in age from 14 to 22 years, were designated as at risk on 
almost every dimension, including over-age for grade, high truancy or suspension rates, 
below grade level on basic courses, substance abuse, and pregnancy. Both students who had 
already dropped out and those who were in danger of leaving school were recruited by werd- 
of-mouth, through radio and television spots, newspaper articles and notices, and flyers 
distributed at area gathering places (laundromats to pool halls). In addition, once students 
were id',nafied as the most at risk in their own home schools, faculty, administration, and 
counsel •>ts referred them to CAP. Some referrals came from the legal system, with juvenile 
authorities working closely with the district schools to find optimum placements. 

The enrollment of every student required the review and consent of both the sending 
principal and the receiving principal. Students who were expelled from their home schools as 
dangerous or violent could be accepted into CAP after extensive review and mutual 
agreement among all parties, but they were not considered for acceptance when their needs 
for assistance were ones that CAP was unprepared to provide. Transportation was provided 
to all students, and many were bused daily from as far away as 40-60 miles. In 1989-90, 
CAP served a total of 145 students on a full-time basis. Approximately 66 of these students 
continued in the program in 1990-91. Most CAP participants were white, while the total 
enrollment of Hispanic students in the program was approximately 40%. 

Project services. Students who attended CAP High School stayed an average of 30 
weeks and received all-day, every-day services during that time. CAP was a comprehensive 
high school serving students from 9th through 12th grades, but it also provided links with 
services required by its adult students. More than a dozen students were single parents and 
learned child-care skills in the vocational program in which their own <;hildren received care. 
Almost one-third of the students were eligible for and worked through a JTPA-funded 
training program. Although over 40% of the students were Hispanic, CAP was unable to 
provide bilingual or ESL education; home schools had the trained staff for those classes and 
for special education programs. 

Parental involvement was a stated objective of CAP, but only a small group of parents 
participated as advisory board members, attended open house and back-to-school nights, and 
conferred witii teachers. In addition to the academic, vocational, and personal counseling and 
guidance services provided by the school counselor, the entire staff informally assisted the 
students in finding solutions to many problems, such as obtaining food stamps, housing, or 
AFDC stipends. Drug and alcohol abuse was a major concern, and counseling services 
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included referral and placement in off-site rehabilitation clinics and after-school Alcoholics 
Anonynx)us meetings. Because some students were emancipated minors, or completely 
independent from their families, the school and staff served often in place of parents. 

Notable Changes in Operations During 1990-91 

In 1990-91, only one major change occurred — paid- work opportunities on campus were 
discontinued as a result of action on the part of the Texas Education Agency. However, CAP 
did add a vocational building during the year that included a library, space for instruction in 
small engine repair and construction trades, a dark room for a photojournalism class, a place 
for an office adimnistration program, and a print shop. This addition enabled the school to 
expand its vocational training offerings to students. More than half of the graduates at the end 
of the second year of program operation were reported to be employed. In an area with high 
unemployment rates, particularly for minority youth (26% for ages 16-19), this seemed a 
distinctive achievement. 

Status in 1991-92 

ITie Cooperative Alternative Program continued its operations in 1991-92 with funding 
from the seven founding school districts and from state compensatory education programs. 
(An eighth district joined the consortium for 1992-93.) The districts not only provided 
financial support for CAP, in the words of the project director, they also began to view the 
program as a viable option for more students. For example, one district that had planned to 
send 10 students sent 35. The school counselor believed that CAP was doing a better job in 
retaining its students, citing improvements in attendance during the 1991-92 school year. 
A major factor in this improvement, in the counselor's opinion, was that students were being 
held to stricter standards, with credit denied in some cases, and were becoming more 
cognizant of the importance of being in school. CAP received several small grants that 
enabled it to provide day-care on campus, making attendance easier for students with 
children. An assistant principal selected from among CAP teachers was named for the 
program, and in 1992-93 he was named principal of CAP. 

Project Outcomes 

All students who were enrolled in CAP during 1989-90 and were between the ages of 
14 and 19 years were selected for the evaluation (N=102). (CAP is and was in 1989-90 
serving students up to 32 years of age.) One hundred students at a neighboring high school 
who also met the state criteria for being at risk according to the CAP counselor and met the 
demographic requirements of the matching strategy were selected as a control group. 
Analyses of baseline differences between these students revealed that CAP students had 
recorded one-point higher grade-point averages for the year prior to participation in the 
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program (i.e., 2.02 versus 1.02). The difference in recorded absences between the groups 
during the basehne year was not statistically significant. Virtually complete data were 
available for comparing average GPAs during the baseUne year, and data for approximately 
80% and 60%, respectively, of the student samples were available for comparing absence 

rates during this period. 

In 1989-90, the implementation study concluded that increased self-esteem and 
improved attendance and academic performance were possible short-term outcomes of the 
CAP program. An increase in student persistence to graduation was suggested as a 
possible future outcome, as was improved labor force participation. 

Before considering specific findings in each of these areas, it is imponant to note that 
the numbers of dropouts from both the CAP and the comparison samples were quite high 
during the evaluation period. For example, by 1991-92, just about one-half the students from 
each group had dropped out or had been expelled or moved without confirmation of re- 
enrollment. SignificanUy, for the analysis of survey data that were collected once at this site 
in April 1990, as many as 24 CAP students and 39 comparison students who would otherwise 
have been expected to respond may have already dropped out (or moved or been expelled). 
In the same vein, records data collected during the 1990-91 school year at this site would not 
have been available for as many as the 24 CAP and 39 comparison students who had been 
reponed as dropped from school during the previous year. While it is the case that a select 
few of these students re-enrolled in school in 1990-91, the overwhelming majority (i.e., all 
but four) remained as dropouts or "missing" students. 

Student self-esteem. In 1989-90, self-esteem data were collected once, in the spring. 
The mean scor« for CAP participants on the 10-item Rosenberg scale was higher than that for 
the comparison students, a difference that was very neariy statistically significant at p < .05 
(i.e., p = .0559). Of course, without baseline data, it is difficult to attribute the higher scores 
of CAP students to pix)gram activities or experiences. In addition, survey response rates were 
at just about 60% and 50% of the original sizes, respectively, for the two samples, so the 
representativeness of this result might be questioned (although analyses based on data 
adjusted for non-response produced similar results). 

Student attendance. The absence rates of both the CAP students and the comparison 
students increased between 5% and 6% during 1989-90 (i.e., about 1 1 days for CAP students 
and 9 days for comparison students); in 1990-91, they both increased by roughly 11% as 
compared to the baseline, or ptc-CAP year. (Note that in 1989-90 the rates of records data 
availability for gain-score analyses of absences were 80% and 60%, respectively, for the 
CAP and comparison samples. Note also, however, that taking into account the 1989-90 
dropout rates for the students in both groups, the rates of records data availability on 
absences for the remaining students in 1990-91 were only 49% and 43%, respectively, for 
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the two groups.) In addition, when they were surveyed in 1989-90, CAP students were no 
more likely than comparison students to report they were doing better in getting to class on 
time. In short, there is no indication that CAP increased students' daily school participation. 

Student academic performance. In 1989-90, CAP students significanUy outperformed 
comparison students academically, improving their CPAs by almost one-half of a grade-point 
over the baseline year compared to a decrease for comparison students of about one-tenth of 
a grade-point for the same period. (Tnese results are based on complete records data for CAP 
smdents and on records data obtained for 88% of the comparison students.) Similarly, in 

1989- 90, CAP students more frequently reported they were doing better in working on class 
projects than did comparison students (i.e., 38% vdrsus 15%). In 1990-91, however, these 
CPA gains disappeared for both groups. In contrast to 1989-90 findings, CAP students in 

1 990- 9 1 recorded grade-point averages that were only .28 lower than their baseline averages, 
while comparison students recorded averages that were almost .60 lower than their baseUne 
averages. This difference in 1990-91 minus 1988-89 (i.e., pre-treatment) grade averages for 
CAP students and the comparison students was not statistically significant, and adjusting for 
rates of student dropout in 1989-90, this result was based on records data obtained for 59% 
of active CAP students in 1990-91 and for 57% of active comparison students. 

The change from a .49 increase in GPA in 1989-90 to a .28 decrease in CPA in 
1990-91 is large and may be explained by two factors: grading policies and attrition. At 
CAP, as was alluded to in the discussion of 1991-92 changes at the site, the standards 
applied to students grew more demanding as the demonstration "aged." Because a percentage 
system was (also) used in computing grades, such that, for example, an average below 60% 
was an F and averages of 62% to 69% defined the D- to D+ range, any greater leniency in 
assigning percentages in 1989-90 could have easily resulted in higher CPAs; that is, the 
difference between a 69% and a 73% average, for example, would mean the difference 
between a 1.3 and 2.0. At the same time, grade averages among CAP students in 1990-91 
would have been affected by 1989-90 graduation rates (i.e., 20 CAP students either earned 
GEDs or graduated in 1989-90), and by rates of student transfers back to their home 
schools— where grading policies may have been much more demanding. By the end of 
1990-91, for example, 8 CAP students had made these transfers. In sum, the increased grade 
averages of CAP students in 1989-90 must be regarded cautiously. 

Student persistence to graduation. The dropout rates for students at risk in this West 
Texas area were higher, year by year, than the dropout rates for smdent cohorts associated 
with any of the other SDDAP projects serving high school students that participated in the 
evaluation. In 1989-90, the dropout rates for school-classified dropouts among the treatment 
and control groups at this site were 14.7% and 21.2%, respectively. The more inclusive 
dropout rates for these groups were 23.5% and 39.4%, respectively. By 1991-92, the rates of 
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school-classified dropouts were 40.9% and 47.2%, respectively, and the more inclusive rates 
were 50.5% and 53.9%. In sum, three years after the student cohort for the evaluation was 
selected at the Coleman, Texas, site, more than one-half of the students were not enrolled in 
school and had not graduated. 

Comparisons of these rates for CAP participants and the control students suggest a 
pattern of differences indicating the effectiveness of the CAP in keeping students in school. 
In both 1989-90 and 1990-91, dropout rates for CAP students based on school-classified 
dropouts together with moved and expelled students were lower than the corresponding rates 
for comparison students. The resulting retention rate for 1989-90 was close to the 80% 
retention goal set by CAP organizers at the outset of the program. (The CAP project was one 
of two SDDAP demonstrations— the other was the Des Moines New Horizons/School- 
Within-a-School combination— that reported having established numerical school retention 
goals.) 

In addition, the dro^wut-only rate for CAP students in 1990-91 was lower than the 
corresponding rate for comparison students, although this difference only approached 
sutistical significance at the .05 level (i.e., p=.085). Finally, survival analyses examining the 
relative lengths of time to dropout for the two groups over the periods from 1989-90 through 
1991-92 show significant effects favoring CAP students. For either school-classified 
dropouts or the more inclusive definition of dropouts, these analyses suggest CAP kept 
students in school longer. 

Student labor force participation. The evaluation did not collect employment data 
from students beyond the initial survey that was administered in 1989-90. A relevant proxy 
measure for one aspect of (prospective) successful labor force participation, however, is 
provided by die combined rates of graduation and GED award. Looking at these data 
(controlling for race/ethnicity, gender, and age differential), we found no differences between 
CAP participants and the comparison students by the end of either the second- or third- 
follow-up periods, which suggested that CAP participants were no more likely than 
comparison students to be in jobs that required evidence of high school completion. By June 
1992, for example, 21% of the original ^.AP sample had either graduated or earned a GED, 
as compared to 29% of the comparison students. 

Other findings. In staffing the program with experienced teachers who wanted to work 
with students at risk, and by providing support to these teachers, the CAP did succeed in 
providing students with a more positive school climate. As indicated by student responses to 
several items on the 1989-90 survey, CAP students reported that their teachers were more 
caring and that the discipline administered at CAP was fair. In the words of one of the CAP 
counselors (and our field staff in 1989-90), the CAP had created a family atmosphere. 
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On these self-evaluations of school climate, 72% of the treatment students agreed with 
statements that teachers at CAP seemed to care about them (i.e., their teachers were 
interested in students, praised their effons, did not put them down, and really listened), while 
only 39% of the control students agreed with those statements. Also, 54% of the treatment 
students agreed with statements that discipline was firm and fair at their school (e.g., rules for 
behavior arc strict, discipline is fair); only 32% of the control students agreed with those 
statements. 

At the same time, the percentages of treatment students reporting parent involvement in 
their school activities were the lowest of any of the projects serving high school students, 
probably because CAP High School was in a remote area, some distance from most students' 
residences. (These reports of lack of parent involvement did not differ significantly from the 
reports of control group students.) 

Factors affecting realization of outcomes. Perhaps because CAP was the product of a 
seven-district consortium that seemed to prefer exerting loose authority over the program, the 
CAP faculty and staff were free to create a close-knit, caring atmosphere for students. 
Because of this and because of child-care and vocational programs that appealed to students 
only marginally inclined to remain enrolled in school, CAP accomplished its goal related to 
student retention in school. The family atmosphere at this demonstration was enhanced 
additionally by the on-campus paid- work opportunities extended to students in 1989-90, 
which, in addition to providing students with money, allowed students to feel closer to the 
paid staff at the site — i.e., to feel they (the students) were (also) meaningful and valued 
contributors to keeping the school functioning. 

Why Some Students Left the Project or Left School 

Exit interviews were carried out wi h a total of 79 students: 30 students who had left 
CAP and school simultaneously, six students who only left CAP, and 43 students who were 
never enrolled in CAP but who dropped out of school. Of the six students who left CAP, 
three did so to return to their regular home school and one moved back with his father. These 
four students reported that they did not feel particular improvements were needed in CAP, 
and all had talked with school personnel in addition to family members prior to making the 
move. The remaining two students reponed they either could not get along with CAP 
teachers and school was boring or they were not learning. These students recommended that 
CAP should enroll more students (i.e., become a bigger campus) and serve better food. 

Of the 30 students who left CAP and school simultaneously, the largest number moved 
with family or with their boyfriend or girlfriend (1 1 students). Seven students reported they 
cither did not like CAP or felt they did not belong in CAP, and six students said they either 
left to get work or to help care for a family member. Four students left either because they 
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were expelled from CAP or had recorded too many absences or suspensions to be allowed to 
continue. Of the remaining two students, one said she left because she was pregnant, and one 
reported that her attendance problem was due to difficulties she encountered in finding 
transportation. The majority of these students talked both with parents, family, or friends as 
well as school personnel prior to leaving (18 students); only four students reported they had 
not talked to anyone prior to leaving school. Twenty-three of the 30 students also said that 
CAP seemed OK to them; that is, in their opinion nothing could have been changed that 
would have caused them to stay enrolled. Five students did suggest that a more diverse 
student body or selection of courses would have helped to change their minds about leaving, 
however. Finally, 13 of these departing students said they would earn a diploma or a GED at 
some later time, while seven students said they did not plan on earning either a diploma or a 
GED. Seven students also expressed uncertainty as to what their educational plans might 
include. 

The 43 students who dropped out of school without ever enrolling in CAP did so for the 
most part because they either were expelled (13 students), did not like school or felt 
themselves behind other students academically (13 students), or moved with family (nine 
students). Five students said they left because of marriage or pregnancy, only one reported he 
left to work, and the remainder gave assorted reasons (e.g., did not feel safe at school, ran 
away, suspended too often). Slightly more than one-half of the students said they talked only 
to persons at school prior to leaving (23 students), and 10 students said they had talked with 
no one. Most of the remaining students who provided this information said they had talked 
with school personnel and friends or family members (seven students) prior to leaving, while 
only one student said she talked only with her friends. 

Surprisingly, about one-half of the students who had dropped out without enrolling in 
CAP said that no improvements could have been made in school to have caused them to stay, 
although such an attuude probably reflected their having given up on school. The remainder 
of students urged thai better teachers be provided, along with greater amounts of personal 
attention and positive attitudes on the parts of both teachers and staff persons. Four students 
specifically added that classrooms made up of fewer students would have caused them to 
reconsider dropping out, and one student noted that she needed child-care facilities at school. 
The majority of these students said they did eventually plan to earn a diploma or a GED; only 
nine students said they had no intention of earning either a diploma or a GED. Of the 34 
students who said they planned to earn recognition of high school course completion, eight 
students specifically said they would either enroll in a GED program or take the test for the 
GED. 
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Diversified Educational Experiences Program (DEEP) 

Freedman Central High School 
Aiken, South Carolina 



Overview of Project Activities, 1989-90 

SDDAP funds supported two versions of the Diversified Educational Experiences 
Program (DEEP) in Aiken, one that operated as part of the regular program in the district's 
middle and high schools and one that was carried out in an alternative school setting for 
middle-grade and high school students. DEEP itself is a model program that was initially 
developed in Wichita, Kansas, and emphasizes a project-oriented curriculum focus. For the 
in-depth evaluation, we regarded the alternative school DEEP program in Aiken as the 
treatment. In addition, since the only students who were regarded as comparable to the 
students in the alternative program (i.e., in terms of personal problems, readiness for 
learning, and general at-risk status) were involved in the regular school DEEP program, we 
selected our control group of students from among the participants in this program. The in- 
depth evaluation at this site thus was intended to contrast the relative effectiveness of the 
alternative school and regular school DEEP program replications in Aiken. For reasons that 
are further elaborated below, this contrast actually became a comparison of DEEP in an 
alternative setting to little or no direct DEEP treatment at all; that is, the failure of the 
regular-school version of DEEP left those students at risk who were identified for services 
pretty much to the regular school curriculum. 

In its alternative school implementation, DEEP at Freedman Central High School in 
Aiken assisted students in earning academic credits and helped them make a transition back 
to their regular schools. It provided a full range of academic courses, including health and 
physical education, career education, and a vocational program in marketing, and offered 
counseling and other support services to students. Additionally, the school had a computer 
lab staffed by a full-time paraprofessional who provided remedial math and reading 
instruction to students who were deficient in basic skills. Students could have also elected to 
take vocational classes at the district's career center. Freedman endeavored to make 
classrooms more hospitable and to help its students learn to cope with an established system 
of rules and to assume responsibility for themselves. The project design called for small, 
intimate classes (with a student-teacher ratio of 10 to 1), whose intent was to create close 
relationships among students and staff. 

Project staff. The design and siti ig of the project was decided by staff at the district 
level, who also hired the principal f : r the alternative school program. Teachers were 
specially selected by the alternative school principal for their compatibility with the school's 
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philosophy, and each teacher, in addition to usual teaching responsibilities, acted as an 
advisor and mentor for seven students. The school's principal recruited staff state-wide 
through advertisements and was responsible for overall planning and administration of the 
project, assisted by a team of district-level administrators. Counseling services were provided 
by a full-time counselor and by part-time district staff. Group counseling emphasized a self- 
help orientation; structured, participatory learning groups addressed social skill needs and 
career information. 

In addition, the project had an advisory committee consisting of representatives from 
the legal justice system, area business and industry, district and school administrators, 
parents, and students. One function of the committee was to educate the community about the 
nature of the project. 

Participant selection. Student eligibility criteria included combinations of the 

following factors: 

♦ Falls two or more grade levels behind in reading 

♦ Has failed two or more grades 

♦ Has received at least three suspensions during the year 

♦ Qualifies for the district' s attendance intervention plan 

♦ Often cuts classes or is tardy 

♦ Participates in few or no extracurricular activities 

♦ Comes from a single-parent family 

♦ Is in foster care 

♦ Is a former dropout 

♦ Is eligible for free or reduced-price lunch 

Students were referred to Freedman by the Board, school personnel, social service or 
legal justice agencies, parents, or the students themselves. Formal referrals came from the 
principal of the sending schools. Student selection was the responsibility of a selection 
committee, which had to reach consensus following a review of records and an interview 
with the applicant and his or her parents. The final step was development of a contract among 
the parents, principal, and applicant that described his or her education-rel&ted goals and 
agreement to participate in project activities according to project rules. 

Freedman High School served all of Aiken County, one of the largest counties in the 
state, along with a portion of adjoining Saluda County. To ensure that the enrollment was 
representative of the district-wide population of students at risk, a quota system was 
established for the district's five administrative areas. 

Being enrolled in Freedman was often a student's last opportunity to attend school 
before being expelled. Many Freedman students had already been labeled as serious 
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discipline problems, and some had already been in the juvenile justice system. All had 
performed poorly in school and had few if any credits. During the 1989-90 school year, the 
project served a total of 116 students, although enrollment was at about 84 toward the end of 
the year. Most of these students were white, although the enrollment of black students was 
approximately 16%. Of the 50 middle school students who completed the year, 20 earned 
promotions and 30 received administrative promotions. Thirty-six were recommended to 
return to their regular schools, while 14 were recommended to remain at Freedman. Among 
high school students, 14 were recommended to return to their home schools, while 20 were 
recommended to continue at Freedman. Most were expected to pass most or all of their 
credits for the year. 

Project services. Instruction was not referenced to particular grade levels; instead, it 
was focused on group projects and was competency-based, featuring use of cooperative 
learning strategies. The varied curriculum, instructional methods, and small-sized classes 
were designed to provide a high level of individualized instruction and suppon. Project 
DEEP students received services full time during their enrollment at Freedman. Average 
length of participation in 1989-90 was approximately 23 weeks. As noted in the National 
Diffusion Network (NDN) literature: 

Project DEEP is an alternative classroom management system that is student centered 
and project oriented. Students in a DEEP classroom identify needs, formulate 
objectives, develop tasks based on these objectives, present group projects based on 
fulfillment of objectives, receive teacher debriefing following the project, and 
participate in their own evaluation. 

In addition to regular middle and high school programs, the school had a formalized 
counseling component, with counseling provided by one full-time counselor and by the 
combination teacher-advisors. The district's supervisor of student services also provided 
assistance in counseling along with other activities (such as attendance monitoring and home 
visits). Other services available to participants included enrollment at the Career Center (the 
district's area vocational-technical school), adjustments in district and state attendance and 
disciplinary policies, and incentive activities. While project staff attempted to initiate 
activities for parents, they reportedly did not experience much success with this component. 

Notable Changes in Operations During 1990-91 

Enrollment at Freedman increased to 129 students during 1990-91. Few significant 
changes were made in the Freedman program during 1990-91; changes were made, however, 
in the regular-school implementation of the DEEP program. In 1990-91, to keep costs at a 
minimum, the training for the regular-school DEEP intervention was provided at Freedman, 
rather than at the various high schools in the district, as was planned. In addition, a 
representative from the Department of Education visiting Aiken in September 1990 
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recommended that additional project supervision and direction be provided for the regular 
school component. In 1990-91, the required administrative support was added to this portion 
of the program. Never that much of a pronounced program, the regular DEEP activity 
received a boost from the district coordinator that rather quickly dissipated as funding for this 
position ended following 1 990-9 1 . 

Status in 1991-92 

Freedman High School continued to operate with funding provided by the district. The 
district-level project director reported being particularly heartened to receive this support in a 
time of reduced resources and attributed it to several factors. The most important was that the 
school, according to the local evaluation, had significantly reduced the dropout rate among 
local students. Its success also seemed to be widely recognized in the community as a result 
of considerable attention from the media, including four lengthy newspaper articles, and 
involvement from community groups, who in turn created additional publicity and support. A 
"Cities in Schools" program also provided credibility and important support for the program 
in 1991-92. 

Project Outcomes 

The evaluation involved a random selection of 95 students from the 1 16 students 
enrolled in Freedman in 1989-90. The selection of these students was made over a period of 
three months, as enrollments in Freedman were staggered largely because of the student 
referral and screening process. Strictly speaking, no students in Aiken or in the surrounding 
area who were not in some sort of specialized program would have been comparable to the 
students who were enrolled in Freedman. For this reason, we drew our comparison group of 
97 students from among the approximately 350 students who were involved in the regular- 
school version of the DEEP program. According to school officials, these students most 
closely resembled the students in Freedman. 

As it turned out, the student-participants in the regular school DEEP program in 
1989-90 may have been exposed to a rather watered down version of this intervention, 
owing to unforeseen problems in selecting and training the required numbers of teachers and 
in scheduling teacher assignments with identified students (i.e., to preserve the teacher-team 
concept). In addition, without a central administrative entity supervising the activities of the 
various school-based DEEP programs, there was little uniformity in the types and intensities 
of services that were delivered to students, and maintaining the enthusiasm of building 
principals for these regular-school DEEP efforts was difficult. Records were lacking as to the 
specific instructional and support services provided to students by the different school-based 
versions of this program. In 1990-91, uncertainty concerning the continued federal funding 
for the programs in the schools once again led to start-up delays in the identification and 
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training of teachers, with the result that, at many sites, school staff actually turned to 
implementing alternative (to DEEP) dropout prevention strategies. The content of these 
programs was not documented by program operators. 

Comparison of the baseline differences in absences between the Freedman and regular- 
school DEEP student samples, based on records data obtained for 80% and 67%, 
respectively, of these samples, revealed that Freedman students were, on average, absent 
more frequently in 1988-89 (i.e., 7% versus 4.8%). This indication of somewhat greater 
disengagement from school was in some way to be expected, as the students sent to the 
alternative DEEP implementation were regarded as one step away from dropout or expulsion, 
unlike those who were retained by their home schools. Insufficient data were available to 
compare the samples on grade averages during the baseline year, more specifically, these 
data for students served in the alternative school setting were not available. 

The chief focus of the Freedman DEEP program was to provide students with intensive 
personal and instructional supports. In 1989-90, our implementation study specialists 
identified two areas of possible short-term outcomes for the program: improved self-esteem 
and improved school affiliation. Lx)nger-term outcomes that were identified included 
improved attendance and academic performance and the possibility of increased student 
persistence to graduation. 

Student self-esteem. Student survey data were collected from Freedman students and 
students in the regular-school DEEP settings in Spring 1989-90. Self-esteem measures 
administered at that time evidence no statistically significant differences between the groups; 
however, the students in the regular-school DEEP program recorded noticeably higher scores 
on the Rosenberg Self-Esteem Scale that approached statistical significance (i.e., p=.07). The 
response rates for these and the other survey items at the site were roughly 65% for both the 
treatment and comparison samples. 

Student school affiliation. Two items from the 1989-90 student survey are related to 
school affiliation: students' perceptions of the importance of school and their participation in 
school activities. With the qualification that baseline data could not be collected, results from 
this survey indicated that Freedman students more often reported thinking that school was 
important. In addition, Freedman students reported greater involvement in social activities at 
school (e.g., student newspaper) than did regular-school DEEP students. 

Student attendance. The rate of absence among Freedman students declined slightly 
(i.e., by almost 1%) from 1988-89 to 1989-90; however, this change in rates represented less 
than one-quarter of the baseline standard deviation. Compari.ng absence rates over time for 
Freedman students and for students in the regular- school DEEP setting, we found no 
differences that were statistically significant (based on complete records data for 80% and 
63% of the student samples, respectively). Similarly, comparisons of the self-reports of these 
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students regarding their getting to class on time since the beginning of the 1989-90 school 
year revealed no differences between the groups. 

Student academic performance. As noted previously, school records data on students* 
grade averages were not available for Freedman students. Examining only self-report data on 
performance in class projects and in reading as compared to the previous school year, we did 
find that Freedman students were more likely than the comparison students to report they 
were doing better. Fifty-two percent of the treatment group as compared to 33% of the 
control group reported doing better on class projects, and 48% of the treatment group versus 
33% of the control group reported doing better in reading books or magazines than in the 
previous year. With data adjusted for nonresponse, the direction of the effect in each case is 
the same; for reading, the result is also statistically significant. 

Student persistence to graduation. Comparison of dropout rates in 1989-90 revealed 
no differences between Freedman and regular-school DEEP students. By June 1990-91, 
however, there was evidence that Freedman students were more likely to have dropped out 
than were regular-school DEEP students, when moved and expelled r.tudents were counted 
along with school-classified dropouts. Although strictly speaking this difference is not 
significant at the .05 level, the (inclusive) dropout rates for the treatment and control groups 
were 36.8% and 21.6%, respectively, with p = .091. By June 1992, 40.7% of Freedman 
students had been classified as dropouts by their schools, compared to 16.5% of the 
comparison students. When moved and expelled students are included, these rates were 
56.8% and 25.3%, respectively. The difference in the dropout-only rates for the samples for 
1991-92 almost reached statistical significance (i.e., p=.0525), and the inclusive rate was 
significant at p< .05. 

Other findings. There was an indication based on unadjusted data (with the direction of 
effect confirmed by analyses of adjusted data) that the parents of treatment students were 
more likely to visit classes than were the parents of control students. This finding may be due 
to the "pull" of the combined efforts of the principal and the teachers or to parents* concern 
that the alternative school is in one of the poorest and toughest neighborhoods in the area. In 
this case, for example, parents may have wanted to visit the school site for reassurance that it 
was a safe and reasonable alternative to their child's regular school. 

Almost two thirds of the Freedman students also reported that the teachers in this 
alternative high school were caring (68%) and that discipline was furo and fair (64%); 
however, the responses of comparison students (who attended other schools) were not 
significantly different. DEEP was planned to create a positive environment for students who 
had felt alienated in other schools (e.g., the careful selection of staff and assignment of staff 
teams to small groups of students), and the positive reports about school climate indicated 
that it may have had some success. 
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Factors affecting realization of outcomes. The presence of a strong and conimittcd 
principal at the Freedman site was critical to the project's organization, training of teachers, 
and gaining the support of parents and central administrators. At the same time, the lack of 
this sort of leadership seems to have negatively affected replication of the regular-school 
DEEP model. Insofar as student outcomes were concerned, there may have been a serious 
mismatch at this site between the DEEP philosophy and instructional approach and the 
interests and abilities of the students who were served. As reported by instructional staff 
during on-site interviews, in several instances the ability requirements of the projects that 
were attempted were much too demanding for the students. In addition, the siting of the 
demonstration at a school previously used as a site for special education may have caused 
enrolled students to wonder whether the project orientation of the DEEP curriculum was 
really some kink of simplified alternative to the regular high school offering. 

Why Some Students Left the Project or Left School 

Exit interviews were carried out with three students who had left the Freedman program 
and one student who had left the regular program. Two students were interviewed upon 
leaving both Freedman and the regular program. Those who left Freedman reported that they 
did so to get a better education (one student), to move with family (two students), or because 
they had gotten pregnant and could not get along with the teachers (one student). One student 
reported that he had only been sent to Freedman for a limited time. Students who left the 
regular program reponed they had been expelled (two students) or had moved with family 
(one student). Three former Freedman students reponed upon leaving that the program was 
"OK" (one said it was "great"), while two students recommended more positive attitudes on 
the part of the teachers or better teachers. One former Freedman student suggested that child- 
care services would have allowed her to remain in the program. 
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Keeping Kids in School (KKIS) 
Ethete, Wyoming 



Overview of Project Activities, 1989-90 

The Keeping Kids in School (KKIS) program served at-risk elementary and secondary 
grade students on the Wind River Indian Reservation and was designed to address some of 
the problems prevalent on the reservation: high rates of unemployment, alcoholism, and 
suicide. Students were referred for early intervention services and for academic tutoring 
outside the classroom. A drop-in center provided a place for students to meet; classes on 
rearing children were offered for their parents, and a social worker provided personal and 
family counseling. Role models were an important feature of the project: the tutors, drop-in 
center supervisor, and project director were all Native Americans. 

The in-depth evaluation focused on the secondary program located at the Wyoming 
Indian High School, which included academic tutoring, activities at the drop-in center, and 
services of the social worker. Academic tutoring was provided to high school participants in 
a pull-out format. The focus of the instruction seemed to be on short-term help, e.g.. 
assistance with a particular assignment rather than skill development more generally. Hours 
spent at the drop-in center were at the students' discretion and were not monitored on a 
regular basis. The social worker was (most) active in 1989-90, when she reported having 
seen about half of the students over the course of the school year. 

Project staff. The tutors who provided academic assistance to the program participants 
were usually college students or recent graduates who came from the same tribes as the 
students (Arapaho and Shoshone). The supervisor of the drop-in center was also a Native 
American local college student. The counselor was a certified social worker. The various 
program components at all sites were managed by the project director, who was also a Native 
American. 

Participant selection. Students were identified for tutoring or counseling by teachers or 
at their own request. Students could choose to attend the drop-in center after school at any 
time. 

Project services. The average project participant received 10 hours of service per week 
for 30 weeks from tutors in pull-out sessions. According to tutors' reports, about half of the 
students in grades 9-12 made use of their services at some time. Typically, students asked for 
help with particular assignments in either math or English; they then went to work with the 
tutors during a free period. Apparently most students used this assistance for short-term help 
only. In interviews with the classroom teachers of tliese tutees there were also complaints 
about tutor activities being disruptive to class schedules and assignments; that is, the 



ERIC 



169 




coordination of teacher and tutor efforts seemed to have been poorly planned for and carried 
out. 

The social worker counseled students who were having either personal or school-related 
problems (e.g., attendance). She was also responsible for such activities as parenting groups 
and Alateen. There was clear evidence that the social worker may have been stretched too 
thin in her efforts with this at-risk student population. 

The drop-in center was intended to be a place where youth could congregate after school 
and do homework. In practice, three of the four rooms were devoted to videos and computer 
games. There was one room where students could study, but apparently it was rarely used. 
Students also went there for snacks. The drop-in supervisor reported that he had seen almost 
all of the students at some time during the year, but the center's location (about a half mile 
from the high school) kept daily attendance down. The students' reports on this differed, 
however; they said no one went to the center because the supervisor sometimes made them 
feel unwelcome. 

According to staff reports, some students participated in all three of the project 
components (i.e., tutoring, counseling, drop-in center). There was little evidence of this, 
however, and no students were found who fit that description. 

Notable Changes in 1990-91 

Only updated reports of students' enrollment status were collected by the evaluation in 

1990- 91; no additional site visits were made and no data from student records were gathered. 
The project director reported that academic tutoring, access to a drop-in center, and parenting 
classes were again offered during 1990-91. Some difficulty was encountered with turnover 
of the tutoring staff, and by the director reported that Native American professional staff are 
difficult to locate and retain. 

Status in 1991-92 

Some of the activities of the Ethete KKIS program were reported to have continued in 

1991- 92, but the tutoring, which was the main focus of the in-depth evaluation, was 
discontinued. The drop-in center remained available to students during after-school hours, 
and classes and other activities for parents were provided with school funds. The project 
director reported that the SDDAP-funded program had some lasting impact in Ethete through 
its work with parents and its emphasis on academic success to improve self-esteem. He 
believed that the project increased the district's focus on outcomes-based education and 
identified needs related to evaluation and accountability: to obtain the technology for 
documenting program activities such as counselor contacts and to provide on-going, on-site 
monitoring. The district's computer capability has reportedly since been improved, and a 
referral system has been implemented to improve service delivery to students. 
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Project Outcomes 

The evaluation focused on student-participants of the high school program, and a 
random sample of 100 of these students was drawn in the fail of 1989-90. One hundred 
control students were selected from the same grades; the candidate pool for the control group 
was made up of students that school officials had identified as in need of tutoring if space 
became available. We attempted to monitor the participation rates of control group students 
in this important aspect of the treatment; however, program records were not kept and tutors 
were not discouraged from tutoring any students (including control students) they deemed in 
need of attention. We are certain that some of the students in the control sample received 
tutoring on at least one occasion, but estimates of the numbers of control students in our 
sample who received these services and for what lengths of time (e.g., only once versus once 
a week for some number of weeks were not available). Although the project director 
recognized the need for improving student service records in implementing future 
interventions, the lack of these data for the period covered by this evaluation makes the 
results of treatment-control comparisons for this site difficult to interpret. The summary of 
project outcomes presented in the following sections should thus be read only as suggestive 
of what may have occurred as a result of KKIS panicipation. 

Measurements of the baseline performance of KKIS and comparison samples do reveal 
differences between the groups. In 1988-89, comparison students had been absent almost 
one-half as many days as the treatment students (i.e., seven days versus 14 days). Data on 
1988-89 grade averages were available for slightly fewer than half the students in either of 
the samples, owing to problems in obtaining student records. However, the mean GPAs for 
the two groups evidence a pattern consistent with baseline absences; namely, comparison 
students had recorded averages that were nearly three-quarters of a grade-point higher than 
did KKIS students. 

Site visitors to the KKIS project in 1989-90 were uncertain both about the chief focus 
of the project and about the potential effectiveness of the rather uncoordinated strategies 
selected to combat the dropout problem. Possible shon-term outcomes that were deemed 
plausible for the project included resolution of students' personal problems and slight 
improvement in basic skills. Future, or longer-term, outcomes were considered to be too 
uncertain to predict. 

Resolution of personal problems. The one-time survey of KKIS students and 
comparison students in the spring of 1990 assessed students' self esteem, how they were 
getting along with friends at school, and whether they had talked to teachers or counselors or 
other adult friends for counseling related to personal problems. KKIS students were no more 
likely to report they received such counseling, and they did not differ from comparison 
students concerning relations with peers at school. In addition, there was no statistically 
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significant difference in self-esteem between the groups. The absence of baseline data on 
these survey items, however, makes it difficult to assess whether KKIS students did 
experience any increase self-esteem or personal abilities to make friends at school. 

School attendance may, to an extent, reflect one's coming to terms with personal 
problems. The more serious the problems, the greater the possibility that a student may turn 
off to school and not choose to participate. Absence data unadjusted for nonresponse 
indicated that, during 1989-90, KKIS students reduced their absence rates by about 4% over 
the baseline year, as compared to about a 1% reduction for the comparison students for the 
same period. When these data are adjusted for nonresponse, the difference in absence rates is 
no longer statistically significant; however the direction of the effect is consistent for both 
groups (i.e., absences were reduced from 1988-89 to 1989-90). As noted in Chapter 1, 
collection of absence and grade data from this site ceased at the end of 1989-90, owing 
largely to the perceived need for more in-depth study of context and process at this site. 
Budget restrictions prevented such inquiries as part of this evaluation. 

Improvement in basic skills. As previously noted, grade averages for the students in 
Ethete proved very difficult to obtain. With data for 38 and 29 students, respectively, out of 
100 in each group, no differences in the performance over time of the KKIS and comparison 
students were found. The small numbers of available records, however, make these data 
insufficient for drawing any firm conclusions about possible improvements in basic skills 
areas. 

Examining the student survey data collected in spring 1990, we found that a greater 
number of KKIS students than comparison students did perceive themselves to have 
improved in doing their homework — the area in which they purportedly received special help 
from their Native American tutors. (The result based on data adjusted for nonresponse is also 
significant.) At this site, for example, 26% of the treatment students said they were doing 
better on their homework as compared to approximately 14% of the control students 
(unadjusted data). 

On the other measures evaluating perceived performance, KKIS students and 
comparison students responded similarly; that is, KKIS students were no more likely to say 
they were doing better in getting to class on time, participating in class projects, or in reading 
books or magazines as compared to the previous school year. 

Other findings. Dropout rates for KKIS and control students in Ethete were not 
significantly different statistically, in any year from 1989-90 to 1991-92. There was also no 
difference between the groups in length of enrollment, based on results of survival analyses. 
By 1991-92, for example, 1 1.3% of KKIS students had dropped out, compared to 6.1% of 
the comparison students. 
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Factors affecting realization of outcomes. KKIS seemed to be a poorly coordinated 
intervention that could not provide students with a sufficiently powerful program to affect 
dropout behaviors. The use of Native American tutors was apparently the most consistent 
service provided to students and may have helped promote school affiliation. At the same 
time, poor coordination of this activity with the work of classroom teachers led to teacher 
resentment of students being pulled out. Greater communication involving both teachers and 
tutors may have helped the demonstration to begin to achieve a sense of shared purpose 
among these key individuals at the site. 
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Discovery Program, North High School 
Denver, Colorado 



Overview of Project Activities, 1989-90 

Following upon the apparent successes in 1989-90 of the Lake Middle School SDDAP 
demonstration (see Chapter 4), another Denver site that was implementing dropout 
prevention strategies patterned after those in use at Lake was included in the evaluation in 
1990-91. North High School in Denver historically had reported high rates of school 
dropout, and school officials believed that an important source of the problem was the lack of 
assistance to 9th-grade students in making the transition from middle school. To address this 
problem, SDDAP funds were used to change somewhat the school experience for 9th- 
graders; groupings, or "families," of students were formed for core classes, and teacher and 
support-staff teams were put together to provide more individual attention to students. Unlike 
at the Denver-Lake program, block, or flexible, scheduling was not an official pan of the 
Discovery program model. 

The stated goals of the program were to improve academic achievement, facilitate better 
adjustment to the school setting, and increase student identification with school norms and 
educational goals. To add this project to the in-depth evaluation, a treatment group was 
identified in September 1990, when the Discovery program was beginning its third year at 
North High School. 

Project staff. In some cases, teachers opted to join the Discovery team; however, it was 
more typical for staff to be assigned to the project. There were team meetings prior to the 
start of instruction and regular team meetings throughout the school year. During these 
sessions, teachers reviewed the progress of the students they shared and prepared for 
meetings with students and their parents, which were held on various occasions throughout 
the school year. Working with the teachers were a counselor, a community liaison (who also 
served as a student advocate and tutor), and a clerk- secretary. Students on each team had 
their four core classes together and took three electives with other students. 

Participant selection. In the spring, feeder middle schools identified incoming 9th- 
graders whose achievement and behavior records were believed to place them at risk of 
dropping out of high school. Students considered most at need were selected, and the 
community liaison, who was a bilingual member of the community, contacted each student. 
In home-visits over the summer, the community liaison (or, in some cases, the counselor) 
explained what the Discovery program was, what students and parents could expect to gain 
from it, and what they could expect from their experience at North High School, thus the visit 
served as both an orientation to the high school and an introduction to the program. 
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Participation in Discovery was optional; both students and parents had to choose it. The 
majority of the students served (i.e., about 88%) were Hispanic. 

Project services. The 120 students who chose to participate were assigned to one of two 
teams. Four core teachers (English, social studies, mathematics, and science) were assigned 
to each team and given a common planning period so that they could discuss the progress of 
individual students, plan strategies to assist them, and meet with parents as a group. 
Emphasis was placed on building self esteem, with regular recognition of achievements 
through bulletin board displays, badges, and other awards, and on involving parents in their 
students' education, with frequent parent conferences and parent invitations to parties 
honoring student achievements. 

Status in 1991-92 

Denver's SDDAP-funded Authentic School Program included a number of components 
and only some continued to be funded in 1991-92 by the district. The Di-overy Program at 
North High School, the focus of our evaluation at the high school level in Denver, was 
expanded in 1991-92 to the lOth grade, but support services were somewhat reduced. The 
community liaison on the project remained, but a counselor was no longer part of the 
Discovery staff. The project director reported that the practices "had been very much 
institutionalized" because they were obviously good practices for children. 

Project Outcomes 

Because participation in the Discovery program was voluntary and both students and 
their parents had to agree to take part, we chose another high school with a similar student 
population as our comparison. More specifically, 99 students were selected at random from 
among the Discovery students, and 1 14 students were selected ffom a comparable pool of 
at-risk 9f'vgrade students at a second Denver high school (West High School, in the city). 
Studenis in this other high school had not had the opportunity to participate in Discovery, but 
beginning in 1991-92, they were exposed to a variety of new (for the school) educational 
initiatives, including a magnet program in international studies, expanded and upgraded 
bilingual instructional programs, and revitalized mathematics, English, and language 
development courses. (In fact, the project designer for th° Lake Middle School 
demonstration— i.e., the proposal author— moved to West High School as assistant principal 
in 1990-91 and reportedly was, in large part, responsible for these new programs.) 

Baseline measures of student performance were obtained from school records and from 
a student survey that was (first) administered in early fall 1990. Despite the fact that the 
comparison sample was selected from a pool of students identified as comparably at risk of 
school dropout as the Discovery participants, the two groups of students differed importantly 
on recorded and reported measures of performance. For example, based on virtually complete 
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data for both samples. Discovery students had recorded greater numbers of absences during 
the baseline year (i.e., an average absence rate of 9.9% in 1989-90 versus 7% for the 
comparison students). They had also recorded lower grade averages during 1989-90 (i.e., 
1.87 versus 2.47) and just about twice the number of total suspensions (i.e., an average .of 1.7 
suspensions versus .86 suspensions). Similarly, on all four of the self-reported baseline 
indicators relating to school performance (i.e., getting to class on time, working on class 
projects, doing homework, and reading books and magazines), comparison students rated 
themselves higher than did Discovery students. (These baseline survey data were also 
virtually complete for the two samples.) Finally, although student attitudes about the 
importance of graduation from high school were more similar (i.e., both groups said it was 
"very important"). Discovery students reported they were less certain than comparison 
students that they would actually graduate. Clearly, these indicators suggest that the study 
samples were not comparably at risk of educational failure. 

We examined possible motivational differences between the two student samples in 
terms of their electing to participate in a Discovery-like program. The five items included in 
this survey covered (actual and hypothetical) parental and peer support for participation in 
the program, along with students' views on how the program might help students (in general) 
feci a part of the school and do well in school and how the program might help them (i.e., the 
respondents) to do better in school. Results from a comparison of these responses revealed 
that, in general, students in the comparison high school reported consistently more positive 
attitudes towards a Discovery-like program (e.g., 82% of comparison students reported 
strong prospective parental support as compared to 60% of Discovery students; 55% of 
comparison students reported strong prospective support from friends for participation as 
compared to 36% of Discovery students). The two groups were similar only in respect to 
their views of the program as important to their own success (as differentiated from the 
success of students generally). 

These results are interesting less because of their value for determining any selection 
bias and more for the way they underscore the performance differences between the groups. 
The students at the comparison site, for example, may have reported greater parental and peer 
support for their participation because, as they were already achieving at higher levels, they 
may have been already experiencing greater parental and peer support for educational 
success. Similarly, comparison group feelings that a Discovery-like program held the 
promise of helping students with their school work and increasing their school affiliation 
may have been the result of their own experiences with school-related successes. The 
similarity of respondents' perceptions that Discovery might help them in school may merely 
reflect the feeling of each of the groups that they had room to improve, albeit, perhaps, from 
different starting points. 
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The initial implementation study visit to this site was made in spring 1991. 
Observations, interviews with staff, and informal discussions with students during this visit 
suggested several areas in which project achievements might be recorded. First, as a result of 
staff efforts to celebrate student achievements (e.g., hold special ceremonies with parents in 
attendance), we judged that students may have developed improved, positive images of 
themselves and their abilities (i.e., increased their self-esteem). Second, because both the 
Discovery outreach worker and counselor made energetic efforts to encourage student 
participation in various school activities (e.g., athletics, clubs), we expected that students 
might feel more involved with school activities as a result of program participation. For 
example, when a student indicated her interest in joining the high school swim team along 
with her re,luctance to approach the coaching staff, the Discovery counselor talked with the 
coaches and accompanied the student to an initial meeting. Such efforts, in our view, should 
lead to increased school affiliation. 

Third, although parents were not consulted on matters pertaining to organization of the 
project (e.g., the make-up of teacher teams, timing and frequency of parent meetings with the 
teacher team), concerted efforts were made to involve parents in conferences with teachers 
and in informal visits to project-sponsored events; for example. Discovery picnics and 
awards nights. During meetings with the teacher teams, the community outreach worker 
served as an advocate for the parent, sitting at the parent's right hand and often assisting the 
parent in responding to questions and comments. In providing the clear impression of being 
interested in parents' participation in project activities. Discovery seemed likely to increase 
the extent of parental involvement in school-related activities. Finally, because of the 
efforts of the outreach worker, the counselor, and the team members to monitor students' 
attendance closely, we judged that the rates of absence of Discovery students were likely to 
decrease. 

Our discussions wiili members of the teacher teams and our sense of the amounts of 
administrative involvement and support of the project did suggest that, beyond the direct 
efforts of the counselor and the community outreach specialist, the project would likely have 
limited effects on students' attitudes and perfonnance in school. Teacher team members 
seemed not to have been well-prepared and supported in efforts to work with this sort of at- 
risk student population (e.g., one teacher commented that she was trained and experienced in 
teaching only advanced English courses at the high school level). Similarly, it was not clear 
that the buiiding principal or the senior administrator overseeing special programs provided 
tangible support to the day-to-day functioning of the program— specifically, support and 
encouragement to the counselor and the outreach worker. It seemed to us likely that teachers 
participating in the programs may have been made more aware of the problems and life- 
situations of Discovery students because they were required to participate in team reviews of 
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students and in team meetings with parents. It was much less clear that the successful efforts 
of the counselor or the outreach worker had affected administrators' perceptions of the types 
of services that needed to be made priorities in school budget planning. 

Student self-esteem. Scores on the Rosenberg Self-Esteem Scale collected in fall 1990 
(i.e., the baseline period) wer*^ just about the same for Discovery participants and comparison 
students. By the end of the first-foUow-up period (i.e., June 1991). the differences in self- 
esteem scores relative to the baseline of the two samples were also not significantly different. 
By June 1992, however, the self-esteem scores of the students who were in Discovery during 
1990-91 were lower relative to the baseline than were the scores of the comparison students. 

Student school affiliation. While Discovery participants and comparison students did 
not differ at the time of the baseline survey administration with respect to their views on the 
importance of school and the importance of high school graduation, they did differ on their 
reported involvement in school activities. More specifically, as measured in the fall of 1990, 
comparison students reported greater rates of participation in academic activities (e.g., 
science or math clubs), social activities (e.g., student council or yearbook), and performance- 
related activities (e.g., varsity or intramural sports). Gain-score analyses of these measures of 
student participation in school-based activities revealed no statistically significant differences 
between the groups by either 1990-91 or 1991-92 relative to the baseline year, indicating 
that the Discovery program had not succeeded in encouraging any greater participation in 
these activities, for example, than was achieved by the comparison students without benefit 
of the program. 

Parent involvement. As noted previously, conceded efforts were made, principally by 
Discovery's community outreach worker, to communicate with parents and to encourage the 
participation of parents in parent-teacher team conferences and project celebrations of 
students' achievements. These efforts appear to have been successful, as our analyses of 
students' first-follow-up minus baseline responses to survey items on parental involvement 
indicate greater gains for Discovery students. Specifically, after having participated in the 
project for most of their 9th-grade experience. Discovery students were more likely to report 
that their parents attended meetings at the schooi ?ind attended school events in which they 
(i.e., the students) played major roles than were the comparison students. In the fall of their 
9th-grade year (i.e., prior to the treatment), the only two differences between the Discovery 
and comparison group studerits in reports of parental involvement was that comparison 
students were less likely to report their parents had phoned or spoken to a teacher or 
counselor and Discovery students were less likely to report their parents had attended school 
events in which they had been featured. 

Student attendance. Students' retrospective evaluations of their getting to class on time 
during the 9th grade year (i.e., the comparison of their evaluations made at the end of the 
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year to those made in the fall) indicated that Discovery students saw themselves as doing 
better than did the comparison students. Data from school records, however, indicated that 
the rate of increase in absences during 9th grade for Discovery students was higher than that 
for comparison students. From their baseline level of 8.9% days absent in the 8th grade. 
Discovery students were recorded as being absent 1 1.9% days during the 9th grade. In 
contrast, comparison students increased their absences by just under 2% from their 8th grade 
rate of 6.5% days absent. By 1991-92, both groups had decreased their absence rates relative 
to the baseline, but the difference in their respective rates was not statistically significant. 

Other findings. Few significant differences that indicate improvements for Discovery 
students were found based on either survey or school records data other than those mentioned 
previously. During the first-foUow-up year. Discovery students did report more frequently 
having talked with counselors about possible careers after graduation than did comparison 
students; however by 1991-92, this difference in the reports of the groups was no longer 
evident. Comparing 1991-92 survey responses to those obtained during the baseline year. 
Discovery students more frequently indicated they thought reading was fun and had talked to 
other adult relatives or friends about planning their high school program. These findings 
present no clear pattern of accomplishments, however, and the rather low (i.e., slightiy below 
50%) response rates make any interpretation difficult. 

School records information on student grade averages revealed no differences between 
Discovery and comparison students; in fact, the CPAs for the groups on average by the end 
of 1990-91 ranged, respectively, from .38 to .28 below their 1989-90 averages. It should be 
remembered, however, that both groups of students were in the 9th grade in 1990-91, and in 
addition to more challenging school work were having to deal with being freshman students 
on high school campuses. GPA information for 1991-92 was simply available for too few 
Discovery students to make any analysis of these data possible. 

Finally, the dropout rates for both groups did not differ significanuy; however, most of 
the students in the two study groups were below the legal age at which they might legally opt 
to leave school. For example, by 1991-92, the dropout rates for Discovery and comparison 
students, inclusive of moved and expelled students, were 17.8% and 1 1.2%, respectively. 

It is interesting to note that, whereas the Lake Middle School project in 1989-90 had 
cvidentiy been perceived by students as having successfully created a more caring, safe, and 
fair environment for learning, this was not found for the Denver-Discovery project. Teacher 
teams were used in both cases; however, the extent of togetherness of the team members, i.e., 
at Lake in 1989-90 and Discovery in 1990-91, was noticeably and qualitatively different. At 
Discovery, for example, it was impossible not to feel as if the teachers on the teams were in 
many cases simply going about their course-related activities as they would have had 
teaming not been in effect. 
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Factors affecting realization of outcomes. The work of the community outreach 
worker and counselor seemed to us to be key elements in the project design. Support for the 
activities of these individuals seemed lacking on the parts of school officials, however, and 
while teachers did appeal' to respect these "ancillary" staff as colleagues, there was little to 
suggest that teachers planned their lessons together with these staff (e.g., taking into account 
their special in-depth knowledge of the students and their families). As noted, the teacher 
teams really were not sufficiendy trained to work as teams and, as important, were not given 
any greater direct control over the demonstration activity (e.g., through more flexible 
scheduling). As a consequence perhaps, these teachers never developed a sense of ownership 
of die program that might have fostered a more accommodating and constructive educational 
climate for students. Finally, there was a lack of recognition of Discovery staff among the 
faculty at the site — being a Discovery teacher had no status and, in fact, had negative 
connotations (e.g., "worst" kids, basement facilities). 




operation Rebound 
Carbondale, Illinois 



Overview of Project Activities, 1989-90 

Established in 1970 by the Carbondale Community High School District, Operation 
Rebound provided a means for area dropouts 16 years of age or older to complete theL- high 
school education. SDDAP funding enabled Operation Rebound to serve more youth and 
provide more services, e.g., transportation, social work services, to youth at risk of dropping 
out. The project was . self-contained (and operated in a wing of a high school for 9th- and 
lOth-graders) and was organized to provide individual instruction and small-group 
instructional or tutoring sessions according to students' needs and instructional preferences. 
Students could enter the program at any point in the year and could attend sessions offered 
during three daytime periods, in the evening, or a combination, to accommodate their 
personal or work schedules. 

The primary goal of Operation Rebound was academic: either completion of credits 
needed for graduation or preparation for the GED. Project staff assessed the individual needs 
and status of each participant and then devised an individualized plan based on the 
participant's academic needs, interests, and personal circumstances. Thus, in general each 
participant worked at his/her own pace, completing courses required for graduation, 
instruction to prepare for the GED, or remedial course work necessary to support return to a 
regular school. Staff also provided career or personal counseling, coordination with needed 
social or health services, and other services to facilitate participants' ability to continue and 
complete educational objectives. 

Project staff. The dii^ctor of Operation Rebound had been with the project since its 
inception. Key staff included certified teachers (called "learning facilitators") in academic 
disciplines, including mathematics, English, social studies, and science. In addition, a 
counselor and student services coordinator provided career or personal counseling and 
referrals to social or health agencies. Staff of the project worked on term contracts, which 
meant that employment depended on continuation of project funding. 

Participant selection. Operation Rebound admitted any youth who was at least 16 and 
had dropped out of school and had been out of school for at least 60 days. In addition, 
students who were identified as being in danger of dropping out could opt to attend Rebound 
instead of continuing at their home school. During 1989-90, the project admitted 460 
students, most of whom were white and all of whom had dropped out of school. Attendance 
varied, with 150-220 students attending the school each day. Students set their own 
schedules, and while some attended regularly and achieved their educational goals in a 
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relatively short time, others tended to drop in and drop out. Students in tiie latter group often 
eventually became serious about tiie high school credential and began attending regularly, 
altiiough some students never finished. One factor tiiat helped to explain variable attendance 
patterns was tiie state's requirement that youth could not take tiie GED examination until 
tiiey reached age 18 and tiieir class had graduated. Thus, for younger dropouts with few high 
school credits, using tiieir time at Rebound to work toward a diploma was not always a 
realistic option, and they waited until they were near 18 to begin preparing for the GED. 

Project services. Students in Operation Rebound averaged 20 hours of service per week 
for 24 weeks; tiie specific amounts of service required for students varied, depending on their 
individual needs. The project's primary focus was academic — eitiier completion of credits 
required for graduation or preparation for tiie GED. Course-related activities at Rebound 
resembled tiie regular high school program, save that students either worked alone or in small 
groups (e.g., from one to three students). Some students completed their credits and received 
diplomas from tiieir home high schools, while others received a diploma from Carbondale 
High School. Because of the multiple needs of many students, the project also provided 
career and personal counseling, proactive referral to community services (particularly health 
and housing services), and transportation to assist students in getting to the school. 

Notable Changes in Operations During 1990-91 

With the exception of possible specific enhancements or the addition of a new staff 
member or procedure, the basic operation of this dropout recovery program had changed very 
little over tiie years. The receipt of SDDAP funding in 1988-89 allowed for tiie program 
enhancements noted above, but during the period of SDDAP funding the academic services 
provided to students remained largely unchanged. 

In 1990-91, Operation Rebound served fewer students than in the previous year (355 
students, as compared with 460 for 1989-90), and the retention rates of students in the 
program were reported to be higher in general. Increased attention was given to improving 
attendance, mainly through the efforts of the counselor and student support services coordi- 
nator and a program to recognize and reward outstanding attendance. Transportation routes 
were also expanded somewhat to cover a wider geographic area. 

Status in 1991-92 

Operation Rebound received a new SDDAP grant in 1991-92 and expanded its 
operation to serve more students and to provide additional services by adding a social worker 
and another teacher. Although Rebound was a well-established program before its 1988 
SDDAP grant, this funding was reported to have enabled it to begin to offer services to a 
larger population covering a wider geographic area. 
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Project Outcomes 

Given this project's primary focus on dropout recovery, an alternative to the matched- 
pairs design strategy was required for the evaluation. We employed a longitudinal study 
design strategy that followed a cohort of 185 9th- to 12th-grade high school students who 
were identified according to local criteria as being most at risk of dropping out of school. 
Records data were collected for all of these students in 1989-90, and surveys were conducted 
with about one-half of the students during the Spring and Fall of 1990. We monitored the 
status of all 185 students closely, recording their moves, if any, to Operation Rebound and 
their subsequent school-related experiences. Follow-up surveys were carried out with 
students who remained enrolled, and special surveys were conducted with students who 
either (1) dropped out of school or graduated or (2) dropped out or graduated from Operation 
Rebound. 

Since the baseline year (i.e., 1989-90), a total of 48 of the 185 initially identified 
students had entered the Rebound program: nine students entered the project after first 
dropping out of school, and 39 students entered via transfers from one of the three high 
schools in the district. Comparing the 1989-90 attendance records and transcripts of the 48 
Rebound students to the 137 students who never entered the program, we found that the 
future Rebound participants were absent more days on average during the baseline year (i.e., 
about five days); however, there was no difference in grade averages between the two student 
groups. These results were the same when dropouts who later entered Rebound were 
removed from the analysis. 

Only about one-fourth of the students who ever entered Rebound completed survey 
forms during 1989-90, so comparisons of responses between Rebound participants and non- 
participants must be regarded with extreme caution. We did examine items related to 
students' plans for school completion, and the results were inconclusive. In response to an 
item inquiring of students how far in school they thought they would get, non-Rebound 
students were more likely to indicate they would attend college. In response to an item that 
asked how certain students were of graduating from high school. Rebound participants more 
frequently indicated a greater level of certainty. 

The chief focus of Operation Rebound and the key longer-term outcome that was 
considered a plausible result of project efforts was the greater rate of persistence of students 
towards completion of high school requirements (i.e., certified either by the diploma or 
GED). This goal applied equally to students who had previously dropped out of school or, in 
a few cases, who needed special assistance to remain enrolled in school. 

A total of three implementation study site visits were made to this project, aU of which 
confirmed that, for students sufficiently motivated to seek out Operation Rebound and to sign 
a performance contract with the project, the individualized services provided would expose 
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them to the required curricula to complete their high school requirements. It should be noted 
that the content of the instruction piovided by Operation Rebound appeared the same as that 
provided to students enrolled in the local high schools, and the individualization of 
instruction was less a tailoring of the curriculum to students' needs than it was an adaptation 
of the typical schedule for providing instruction. 

The atmosphere of Operation Rebound was also much less like the typical classroom 
and more similar to an alternative school setting in that students often worked on their own 
assignments and were free to get up, move around, go outside for a cigarette, socialize, or 
leave and then return at a later time to their work. It is fair to say that while project staff 
organized instructional schedules, transportation arrangements, and support services (e.g., 
child care) to facilitate students' attendance, attendance as well as completion of required 
assignments were viewed as the sole responsibilities of the students. 

Student persistence to graduation or to earning a GED. Operation Rebound serves 
as both a dropout recovery and a dropout prevention activity in Southern Illinois. As a 
recovery program. Operation Rebound aims to provide instruction and ancillary services to 
students who have already left school and who desire to earn degrees. As a dropout 
prevention program. Operation Rebound serves as an alternative school to which regularly 
enrolled students may transfer to earn required credits or to make up failed courses. In 
evaluating the performance of students served by Operation Rebound, it is important to 
distinguish between these two distinctive service aims. 

Recovery— Of the total 185 students classified as being at risk of failure by high school 
officials in the area, approximately 11%, or 20 students, dropped out of school prior to 
completion or had their schooling terminated as a result of juvenile court actions against 
them. 

Of these students, slightly less than half (i.e., nine students) entered the Rebound program, a 
recruitment rate of 45%. In comparison, two of the 20 dropouts re-enrolled in one of the 
district's three high schools, a recruitment rate of 10%. 

As of June 1992, three of the former dropouts who had enrolled in Operation Rebound 
had dropped out of the Rebound program, two students had earned GEDs while in Rebound, 
and four students remained enrolled in the program. In all. Operation Rebound had achieved 
a recovery rate of 67% (or six out of the nine former dropouts). Both of the two former 
dropouts who re-enrolled in high school (i.e., did not seek assistance from Rebound) 
graduated, giving the three high schools a recovery rate of 100% for dropouts from this 
sample who re-entered. 

Dropout prevention — Approximately 24% of the 185 students initially identified by 
local school officials as at risk of dropping out of school entered Rebound by means of a 
transfer from the regular school program. The majority of these students were attempting to 
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make up or complete required classes, while attending to work and other non-school 
responsibiUties (e.g., child care) off campus. Of these 39 students who entered Rebound, 10 
dropped out of Rebound and failed to return to school, 13 graduated with diplomas, and 16 
remained enrolled as of June 1992. Of the 13 graduates, one student transferred back to the 
regular program and graduated with her class while the other 12 received diplomat from 
Rebound. Of the 16 enroUees, one student had transferred back to the regular program and 
remained enrolled as of June, while all the others remained enrolled in Operation Rebound. 

Based on this student sample, the engagement rate for Operation Rebound was 74%— 
29 out of the 39 students who initially transferred to the demonstration were either graduated 
or remained enrolled as of June 1992. In comparison, the combined engagement rate for the 
three local high schools was 82%; that is, of the 146 at-risk students in the sample who did 
not transfer to Rebound, 1 19 were either graduated, enrolled, or had transferred to another 
high school outside the district as of June 1992. 

Summary— In general, combining the results for recovery and for dropout prevention. 
Operation Rebound provided services to 48 of 185 at-risk students, or 26% of the at-risk 
population identified by school officials in the fall of 1989. Of these 48 students, 13 dropped 
out of the program (and from school as well), 15 graduated with a diploma or received a 
GED, and 20 remained enrolled in Rebound or in the regular program after transferring from 
Rebound as of June 1992. By June, Operation Rebound had achieved about a 31% graduation 
rate while experiencing a 27% rate of dropout among enrolled students. 

In comparison, of the 146 at-risk students who never entered this SDDAP 
demonstration, 47 are known to have earned diplomas from one of the three high schools, 18 
dropped out, 72 remained enrolled or had transferred to high schools outside of the district, 
and nine had moved without evidence of re-enrollment or could not be found. By June, the 
three district high school had achieved a 32% graduation rate while recording overall dropout 
rates of 12% (dropouts only) and 19% (dropouts plus moved students and those whose status 
could not be determined). 

The comparison of Operation Rebound students with students who never entered the 
program can be misleading, regardless of the at-risk status of these students defined in 
1989-90 by local high schools. Students who opted or were transferred to Rebound may have 
been those with the most serious problems in school or those who experienced the most 
intense pressures to leave school to work or raise a family. Direct comparisons with the 
students who remained in the regular program may therefore underestimate the efforts of this 
SDDAP-funded demonstration. 

Factors affecting realization of outcomes. The success of Operation Rebound 
depended, in large part, on the motivation and persistence of students. As noted, the 
interaction that was provided was not different from what would have been provided in the 
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regular school setting, except for the fact it was provided to smaller groups of students. 
Rebound also allowed students to set their own schedules for instruction and permitted them 
considerable freedom during interactional episodes to take breaks and to attend to personal 
matters. 

Why Some Students Graduated or Left the Project or Left School 

Exit interviews were carried out with eight of the 15 students who graduated from 
Operation Rebound. These graduates were asked about the ways in which Operation 
Rebound had helped them in completing the district's requirements and about what 
Operation Rebound offered that the regular program did not. Six of these students said that 
Rebound teachers paid greater attention to their needs than did the teachers in the regular 
program. Three commented that they especially liked working on their own or in small 
groups rather than in larger classes. Three students noted that Rebound's flexible scheduling 
allowed them to woric while they completed school, and two other students said the flexible 
hours made it easier for them to enjoy outside-school activities. Selected comments included 
tlie following: 

» Unlike regular school, where students can get lost in the system, at Rebound the 
teachers paid attention to student needs, and it made learning fun. 

• When you have questions, in Rebound they get answered better, it makes the 
assignments easier to understand. 

• The hours and the classes let you work while you go to school; if regular school was 
set up like Rebound I wouldn't have had as many problems as I had in school. 

• The teachers were better in Rebound; I didn't f::el stupid there. 

• Rebound teachers were a little strict, but I liked working by myself and being able to 
take a break when I needed one. 

• I did miss playing sports, but all in all, I liked the free time I had after finishing 
Rebound assignments. 

• Rebound made it possible for me to get into the classes I needed; because of tiiis, I 
was able to graduate at about the same time as my friends did. 

Interviews were also conducted with 10 of the 16 students who dropped out of 
Operation Rebound. These students were asked why they left the program, what might have 
encouraged them to stay, and what their current plans were with respect to finishing high 
school. They were also asked who they talked to about leaving the project before making 
their final decision. Three of these students reported they dropped out because they either did 
not like the teachers (e.g., they thought the teachers were too demanding) or just were not 
interested in doing the work. Three said they left because they had to work for pay; two 
because they either were or were soon to become fathers and had to work to support their 
families, and one who said that he wanted to learn a trade in the service. Two students had 
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run afoul of law enforcement authorities; one was dropped due to excessive absences and one 
left to enter a drug rehabilitation program. Finally, one student ran away from her family and 
left the area, and one other student left the area along with his family. 

The three students who left because of what they perceived as excessive restrictions and 
less than interesting assignments suggested that the project needed more understanding 
teachers who pushed less and listened more, fewer rules, and less homework. One of the 
students who left to work stated that he thought the program was great, and he hoped it 
would still be available when his baby was older so he could return and finish. Of the four 
students who responded to questions concerning their future plans regarding school, two said 
that maybe they would return in the future, one said he definitely would return to finish, and 
one noted that he had no plans at this time. Most of tiiese students either did not talk with 
anyone about their plans to leave school, or discussed them with friends or a family member. 

Finally, we interviewed 17 students who had dropped out of the regular program. The 
most common leason given for leaving school was tiiat students could not keep up with 
course work (seven students). Other reasons included persistent problems in getting along 
witii teachers and other students (three students), having been expelled (two students), and 
getting pregnant or having to raise a family (two students). Nine students reported having 
talked to someone at school about leaving school, and four students suggested that more 
informed administrators, changes in school administration, or more help and encouragement 
ftom teachers may have changed tiieir minds about leaving. Approximately one-third of these 
students (five students) said they had heard of Operation Rebound or knew there was such an 
alternative program available in the area. 
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Summary and Interpretation of Findings for 
High Schoo] Projects 

Summary of Findings 

Table 16 presents findings for the seven high school-level demonstration projects that 
participated in the evaluation. As indicated on this table, at one site— Memphis— two student 
cohons were selecied; at the Denver-Discovery site, only a 1990-91 student cohort was 
followed. We chose the Memphis site as one to study with a second cohort of students 
because preliminary implementation study data suggested the intervention to be especially 
promising. The Denver-Discovery site was selected for study in 1990-91 because it promised 
to replicate a design that was developed with SDDAP funds and appeared to be working 
successfully at the middle school level (i.e., at Lake Middle School, see Chapter 4). 

Table 16 presents results based on data drawn from school records for the years 1989 
through 1992. Analyses of absence and grade average data compared follow-up data to 
performance at baseline levels for both treatment and comparison samples. Dropout-rate 
comparisons utilized rates based on both school-classified dropouts only and these dropouts 
added together with expelled students and those who moved during the period and were 
never confirmed as re-enrollees. All analyses of records data compare changes in 
performance over time between samples, and determinations of improvements for the 
treatment samples are always made in this relative context. Table 16 also presents results 
from student surveys; in the cases of the 1989-90 student cohons at the sites, improvements 
for the SDDAP participants are determined by comparing responses of treatmejil and control 
samples at one point in time— Spring 1990. For the 1990-91 student cohorts, changes in 
survey responses over tiine relative to the comparison groups determine whether SDDAP 
participants have "improved." 

On this table, the results from the Des Moines, Coleman, and Memphis projects stand 
out, particularly the findings related to lower dropout rates. It is noteworthy, for example, 
that these three demonstrations each made use of paid-work incentives and provided students 
with strong career preparation and vocational exploration opportunities. In addition, the 
findings at the Coleman site pertaining to school climate (e.g., more caring teachers) are 
important and suggest a theme similar to the one indicated by the findings presented in 
Chapter 4 for the Denver-Lake project. At each of these three high school sites, whether 
through lower absence rates, higher grades, and/or lower dropout rates relative to a 
comparison group, students participating in the demonstrations evidenced greater interest in 
school participation. 
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Table 16 

Summary of Improvements for Students Served in 
the Seven SDDAP High School Demonstration Projects: 
Measures Based on School Records and Student Self Reports^ 



Des Moines, lA 
1989-90 Student Cohort 
First- Year Follow-Up 

• Higher grades fsr) 

• Lower dropout rates ( sr) 

Second- Year Follow-Up^ 

• Lower dropout rates ( sr) 
Third-Year Follow-Up^ 

• Lower dropout rate'^ ( sr) 



Coleman, TX 
1989-90 Student Cohort 
First- Year Follow-Up 

• Higher grades fsr) 

• Lower dropout rate^ (sr) 
I • More caring teachers fss) 

• Fairer discipline (ss) 

• Better at working on class projects 
compared to previous year (ss) 

Second-Year Follow-Up^ 

• Lower dropout ratc^ ( sr) 
Third-Year Follow-Up^ 

No improvements found 



Memphis, TN 

1989- 90 Student Cohort 
First- Year Follow-Up 

Fewer absences^ (sr) 
Second-Year Follow-Up^ 

Fewer absences^ (sr) 

Third-Year Follow-Up^ 

Lower dropout rates (sr) 

1990- 91 Student Cohort 
First- Year Follow-Up 

• Fewer absences (sr) 

• More frequently talked to counselor(s) 
about high school program (ss) 

Second- Year Follow-Up 

• More ambitious plan for school 
completion (ss) 



Aiken, SC 
1989-90 Student Cohort 
First- Year Follow-Up 

• Better at working on class projects 
compared to previous year (ss) 

• Better at reading books or magazines 
compared to previous year (ss) 

• Parent(s) visited classes ( ss) 
Second- Year Follow-Up^ 

• No improvements found 
Third-Year Follow-Up^ 

• No improvements found 
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All estimates of improvements for students in the demonstrauon projects are relaUve to comparison students; 
that is. means or rates assessed at one point in Ume or gain-scores for students in the demonstrauons are 
determined to be improvements by comparing them to similar measures for comparison students. Absences, 
grades, suspensions, and dropout rates are based on school records (sr); all other measures are derived from 
student surveys (ss). Survey data for the 1989-90 student cohorts are based only on one admmistration of the 
survey instrument Complete informaUon for all findings is provided in Appendix A 
Only possible improvements in absences, grades, and dropout rates were assessed for this follow-up. as 
student survey data were not available for analysis 

Only possible improvements in dropout rates were assessed for this follow-up. as neither student survey data 
nor other records data were available for analysis 

TUis rate is based on school-classified dropouts together with moved and expelled students The dropouiratc 
for the comparison group at this site in 1992. calculated excluding moved and expelled students, w^ 17%. 
compared to a 30% rate for the comparison group; the level of significance for this difference was .059 
TTiis result is based on less than 50% response rate for the comparison sample at the site 
This rate is based on school-classified dropouts together with moved and expelled students 
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Tabk 16 (Cont'd) 
Summary of Improvements for Students Served in 
the Seven SDDAP High School Demonstration Projects 
Measures Based on School Records and Student Self Reports 



Ethete, WY 
1989-90 Student Cohort 

First- Year FoUow-Up 

• Fewer absences (sr) 

• Better at doing homewoik 
compared to previous year (ss) 

Second-Year Follow-Up2 

• No improvements found 
Third- Year FoUow-Up3 

• No improvements found 



Denver-Discovery, CO 
1990-91 Student Cohort 

First- Year FoUow-Up 

• Better at getting to class on time (ss) 

• More frequently talked to counselor(s) 
about possiWe careers after graduation (ss) 

• Parent(s) attended school meetings (ss) 

• Parent(s) attended special event on 
campus in which student was featured (ss) 

Second- Year Follow-Up 

• More frequently thought that 
reading is fun (ss) 

• More frequently talked to other adult 
relatives or friends about planning 

high school program (ss) 



Carbondale, IL^ 

1989-90 Student Cohort 

• 45% of dropouts re-€nrolled by 1992 (sr) 

• 67% of dropouts who enrolled since 1989 either earned GEDs or remained 
engaged in school as of 1992 (sr) 

• 74% of snrolled non-dropouts since 1989 (i.e., transfers from the regular program) 
remained engaged in school as of 1992 (sr) 

_,^_^i%.o£enroUed_droBoutsjn^^ graduated by 1992 (sr) 



A longitudinal rather than a matched-pairs design was used at this site, with the 1989-90 cohort of students at 
risk followed from 1989-90 through 1991-92. Data on enrollment status were compiled based on school 
records 



The findings for the Aiken and Ethete projects are rather sparse, and data collection 
problems at both sites necessarily limited the scope of the evaluation. In neither case is ther« 
a consistent pattern of findings. 

Results on Table 16 for students in the Denver-Discovery project at the time of the first- 
follow-up, for the most part, serve to confirm the (immediate) effectiveness of efforts made 
by*the program to counsel students and to involve their parents in school activities. In 
particular, the efforts made by the project's community outreach worker and others in the 
project (e.g., the counselor) to see that parents attended meetings with teachers and school 
celebrations of students* achievements are shown to have positively affected students' 
perceptions of their parents' level of school involvement. By the time of the second-follow- 
up, however, when many of these students were in the regular program there is no longer any 
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evidence of increased parental involvement, and the two results listed for students are 
unrelated to any previous outcomes. 

Finally, the results for Carbondale's Operation Rebound provide commentary on its 
functions as both a recovery program and a dropout prevention program. As a recovery 
program, the results may be encouraging; close to 50% of the dropouts from local high 
schools were successfully recruited into the program from 1989 to 1992. Of those dropouts 
who did enroll, however, slightly more than one-third eventually dropped out. Of the 39 
students enrolled who were transferred to Operation Rebound since 1989, 26% dropped out, 
compared to either a 10% dropout-only rate or a 14% dropout-plus-moved-student rate for 
those similarly at-risk students in the original sample who never transferred to the program. 
As noted earlier, however, transfers to Rebound may represent special challenges to 
education-related programs; that is, these students may be most at risk of leaving school 
prematurely. Nevertheless, unlike the Coleman demonstration, which changed the high 
school experience for students, or the Aiken demonstration, which altered the curriculum and 
featured cooperative learning. Operation Rebound provided high school transfers with the 
regular school curriculum in a relatively regular-school atmosphere. The most pronounced 
differences between this demonstration and the regular high school programs in the area were 
the smaller class sizes and the flexibility offered to students to schedule their in-class work 
on an individual basis according to their outside-school commitments. Judging from the 
results, these enhancements may not be sufficient to meet the needs and interests of students 
at risk who are perhaps most vulnerable to the sons of pressures that cause high school youth 
to drop out of school. 

Interpretation of Findings 

The tangible rewards of paid work, when linked to school attendance ^nd accompanied 
by a program of careful preparation and follow-up on the job, may provide important 
incentives for at-risk high school students to remain active in school. This seems true 
regardless of whether the incentive is offered directly by the school or by an adjunct 
organization. When paid work is less salient due either to the ages or the extent of problems 
of the youth served or to the unavailability of jobs in the geographic area, problems are likely 
to arise for dropout prevention programs unless different incentives can be identified and 
provided. The organizational structures and strategies employed by the seven SDDAP high 
school projects participating in this evaluation reflect their varying levels of concern for 
finding effective incentives for student engagement in school. 

Organizational structure. The seven high school demonstrations may be divided into 
three groups: regular-school demonstrations, physically- separate alternative-school 
demonstrations, and demonstrations operated by non-school agencies. The effects of 
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organizational structural elements on project functioning are best described within these 
categories. 

Regular-school demonstrations — ^The Des Moines, Ethete, and Denver-Discovery 
projects integrated various student incentives into the regular school program, with varying 
levels of change in the regular school schedule. In Des Moines, for example, the 
demonstration placed students in a school-within-a-school setting where the class sizes were 
small and individualized instruction was emphasized. The key incentive of this 
demonstration, however,, was the placement of students in paid jobs after school on condition 
that they remain enrolled and performed acceptably in their NH/SWS classes. These 
placements were made in largely invisible fashion insofar as the daily school schedule was 
concemed; however classroom preparation for work was an important component of the 
program. Similarly, while the incentive for the student might be the money earned from the 
job, the school worked with each employer to ensure that the assigned job was appropriate 
and would in no way jeopardize the student's remaining enrolled (i.e., employers were 
prohibited from hiring their part-time students on a permanent basis until after the student 
graduated). 

In Ethete, where jobs are scarce, the regular schedule was modified to allow for pull-out 
tutoring of students by Native American tutors; in this case, the opponunity for Native 
American students to interact with their older peers and, perhaps, to discuss a wider range of 
topics than those covered in the classroom by non-Indian teachers constituted an incentive 
for students to remain engaged. Finally, in Denver, where the ages of the students (i.e., 13-14 
years) served by the demonstration precluded a heavy emphasis on obtaining paid-work 
opportunities, the incentives provided to students to encourage their full participation in 
school included personalizing the school experience and providing consistent support and 
recognition of school achievements. The school experience for these students was 
restructured, such that their teachers worked in teams, group spirit among the students and 
teachers was encouraged, and a counselor and an outreach worker were available to assist 
with student problems and family concerns at all times. 

Clearly, the power of the Des Moines incentive was not equaled at either the Ethete or 
the Denver-Discovery sites, as evidenced by the results shown on Table 16. In our view, 
however, this was not due entirely to the greater perceived value of paid work among high 
school students. To an extent, the structural features of the Ethete and Denver-Discovery 
demonstrations, i.e., how these projects delivered incentives to students, may have played 
important roles in constraining student performance gains. Specifically, the Ethete pull-out 
program actually alienated regular teachers at the panicipating school, and the Denver- 
Discovery program required more active effon to ensure that the teacher teams were 
functioning as intended and in a fashion that was coordinated with the ancillary services 
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being provided to the students and their families. Notably, the Denver-Discovery program 
also would have benefited from more pronounced and ongoing efforts to win the support of 
key school administrators and other school faculty, a point we describe in greater detail 
below. 

Unlike the problems with pull-out programs cited in Chapter 4, which were concerned 
with the stigma for students associated with being singled out for special service, the KKIS 
approach to pull-out programs was resented by some teachers who viewed the Native 
American tutors as not supporting classroom aims. As a result, the students who were pulled 
out for tutoring (and looked forward to the tutoring) found themselves regarded by teachers 
as having engaged in activities that were unrelated and in some ways interfering with their 
school assignments. To make this sort of student-directed incentive work, the KKIS program 
needed to work harder with teachers and tutors in trying to build a common frame of 
reference and shared set of goals. Pulling students from classrooms to provide enrichment 
activity that students may enjoy is likely to be counterproductive if the teachers of those 
classrooms see the activity as irrelevant and distracting; in fact, the affected students may 
actually find it more difficult to make up missed class work if teachers resent their leaving 
class in the first place. 

The rather slow start-up of the Denver-Discovery project also underscores the 
difficulties in coordinating diverse project elements with the regular school program. At this 
site during 1990-91, it appeared that the teacher teams were neither provided with sufficient 
orientation into the program nor were they successfully pulled together into teams. To 
complicate matters, the activities of the teacher teams required coordination with the efforts 
made by the project counselor and community outreach specialist. Most difficult of all, the 
teacher team members, the counselor, and the outreach specialist seemed to identify their 
primary assignments as project staff members rather than school staff members. There was 
little effort made by the teachers to win support for the project from other faculty, for 
example, and the project counselor carried on her activities entirely outside of the regular 
counseling function at the school. Perhaps as a result, some teachers talked of their plans for 
"going back" to the regular program at the end of the school year and, at the end of 1990-91, 
the project counselor was re-assigned. It is also important to note that without an effective 
plan for coordinating project activities of these sorts with the regular program, the 9th grade 
students served by the project are likely to be less well-prepared for the transition to 10th 
grade. 

Alternative-school demonstrations— The Coleman, Aiken, and Carbondale projects 
provided physically-removed, alternative- school experiences for highly at-risk students— 
i.e., those who were not succeeding in the regular program and were in greatest danger of 
dropping out. Of these projects, the Coleman effort was the most ambitious in recreating the 
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school experience for students: special teachers and counselors were recruited to the school 
and were provided orientation and encouragement to work as a team in assisting students 
with a wide range of personal as well as academic problems; special facilities were 
developed to offer vocational classes in areas of particular student demand; and paid-work 
opportunities on the school campus were provided to students. To a large extent, the breadth 
of the Coleman demonstration effort resulted from the fact that this project was a joint effort 
of more than one school distric:. Specifically, five local districts pooled their ideas and 
resources to initiate the project in 1988 in a setting that was more or less equidistant from all 
participating districts, and two additional jurisdictions began working with the project during 
subsequent funding periods. Evidently, this type of inter-jurisdictional coordination in less 
densely populated areas is feasible and may be a desirable approach to combating regional 
school dropout problems. The success of the approach was evidenced in 1989-90 by positive 
student perceptions of the school climate, higher grade averages, and lower dropout rates 
(when moved and expelled students were included in the analysis). More generally, based on 
survival analyses over the full range of years covered by this evaluation and regardless of the 
specific dropout measure employed, the Coleman project helped students to stay in school 
longer. The success of the effort is underscored in a somewhat different way by the fact that 
the CAP program has continued with multi-jurisdictional support and without cut-backs in 
staff or programs since the termination of SDDAP funding in 1991; in fact, an eighth district 
joined the CAP consortium in 1992-93. 

The Aiken and Carbondale alternative schools, situated similarly in less densely 
populated areas, continue to be sponsored by single school districts and to focus their efforts 
on students who are not succeeding in other district schools. The Aiken demonstration is the 
more ambitious of these two in attempting to replicate a nationally recognized model 
program within one of the district's high schools to provide students with an alternative 
school experience. Given the staff training and support needed to carry out this 
implementation, it was critical that the demonstration be supported by a strong building 
principal, and such support was provided. In contrast, the regular school-based 
implementation of the same model suffered from a lack of strong central administration. The 
extent of the student problems at Freedman may have largely dampened the positive effects 
of project efforts, as shown on Table 16, although project students did perceive themselves as 
having improved in class work and in reading over the 1989-90 school yecr. 

In Carbondale, the alternative school was and is really an adjunct to the regular district 
program (and is situated in the freshman high school). Nominally, supervision of the 
demonstration is the responsibility of a regular high school administrator; however, the 
program was conceived a number of years ago and has been operated since by staff at the 
alternative site. The project offers the regular school curriculum to students, allowing 
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students to determine their own schedules for instruction. The evaluation results suggest that 
this strategy allowed for recruitment of about one-half of all dropped-out students over a 
three-year period and achieved success (i.e., graduation or continuing enrollment) with two- 
thirds and three-fourths, respectively, of dropped-out students and those students who 
transferred in from the regular program. Permitting students to sign up for various 
instructional schedules that best accommodate their outside-school responsibilities is not 
likely to be feasible in most high school settings, but this organizational approach to meeting 
student needs does seem to be effective with students who want or need to v/otk. This sort of 
strategy combined with a paid-work incentive of the sort described above may work to the 
advantage of both students and program organizers. 

Inter-jurisdictional support seems to have provided the Coleman demonstration with a 
broader mandate and resource base. In addition, perhaps because of this organizational 
arrangement, the Coleman project director seems to have been freer to organize the 
alternative school "from the ground up," recruiting teachers, arranging facilities, and so on. 
In comparison, the Aiken and Carbondale projects seem more limited in focus and may have 
seemed somewhat more limited in scope at the outset to the instructional staff. In Aiken, 
typical start-up requirements and the needs for continuing staff encouragement and assistance 
were made less burdensome by strong building leadership, which seems essential in attempts 
to transport and replicate proven model programs. However in Carbondale, the project seems 
more limited by design and has retained its specific service orientation in large part due to the 
continuing direction of activities by the same management team. 

Although this evaluation has very little information on the stigma and the negative 
effects of homogeneous groupings of students as these may pertain to alternative school 
settings, it is important not lo forget these issues. In the cases of the Aiken and Coleman 
sites, the students transferred to the demonstrations were generally acknowledged as 
problems by their home schools. Specifically, in both cases the transferred students were 
regarded for practical purposes as "getting their last chance." In these cases, it may be that 
stigma is less likely to contribute to the continued poor performance of the students involved, 
since they are akeady likely to have been typecast as problem students before the transfer. 
The issue of homogeneous grouping is an interestii .g one, however, for although these 
students had not succeeded in their home schools, the transfer was effectively placing them 
among students who were very much like them. In fact, in their exit interviews, several 
Coleman students did report they had dropped out because they felt they did not belong in 
CAP, and several others in describing potential project improvements mentioned specifically 
that CAP should seek to enroll a more diverse student body. This factor was less often 
mentioned by those students leaving the Aiken and Carbondale projects, however. 
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In contrast to both the Aiken and Coleman demonstrations, the Carbondale project may 
have been expected to have produced less stigma for students because, to a large extent, it 
merely was offering the high school curriculum to those who had outside-school 
commitments. Similarly, although the students who enterec deration Rebound may have 
shared similar attitude: about school and similar interests in outside-school activities, the fact 
that each student determined his or her own schedule for instruction effectively resulted in 
most cases in randomly assigned classes of one or two students at most in any one subject; 
that is, in Carbondale, students typically were not grouped by ability for instructional 
purposes. 

Demonstrations operated by a non-school agency— In Chapter 3, we surmised that 
special challenges may face projects that attempt coordinated service arrangements between 
community-based organizations and the public schools. In Memphis, this type of 
organizational arrangement may have also contributed to difficulties in improving students' 
grade averages or levels of skills related to classroom assignments. 

In one sense, the Memphis Partners Collaborative (MPC) did successfully improve 
students' school performance by improving their attendance relative to comparison students 
and to baseline levels (i.e., reducing the rates of increase in their absences from one year to 
the next when compared to a control group). MPC staff accomplished this by providing 
students with tangible incentives for good alttendance and by making school visits regularly 
to check on students' attendance patterns. Although MPC also provided incentives to 
students for earning higher grades, they were not as successful. The selection, orientation, 
and continuing supervision of MPC teaching staff for the Saturday program was largely left 
as a responsibility of the Memphis Public Schools. Indeed, transportation to and from the 
Saturday program and the teachers who participated in the classroom instruction phases of 
the Saturday program were the key areas of responsibility of the public schools. 

In our view, the contents of the instruction provided on Saturdays and the methods by 
which these contents were delivered to students left much to be desired. Notwithstanding that 
the teachers were advanced career ladder teachers and were receiving extra compensation 
for this Saturday assignment, many approached their tasks as a sixth day of regular school 
remediation. The small class sizes were not used to maximum advantage, and drill-and- 
practice tasks were often followed by individual students being called on to provide answers 
or to '"go to the board. . .." In addition, career exploration classes were, on occasion, used to 
"help" students see that ambitious career goals were unrealistic — that students should set 
their sights lower, to focus on the types of jobs they might "actually" be able to get. In the 
midst of an upbeat, student-centered dropout prevention activity, these classes stood out as 
being the least constructive element, yet the community-based organizers of the activity did 
not perceive themselves as being responsible (or perhaps as sufficiently knowledgeable) for 
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evaluating the instructional program. Similarly, public school officials, perhaps because the 
MPC teachers were operating outside the typical school environment, did not actively 
• monitor the efforts of the teachers so as to identify areas needing improvement. As a result, 
the apparent problems related to the instructional component of the MPC effort went 
undetected and unresolved. In sum, an inter-organizational planning and review process 
should have been built into the project design, allowing these types of problems to surface 
and be corre<;ted. 

Effective strategies. One apparently very successful strategy for promoting school 
engagement among students at risk of dropping out at the high school level is the use of a 
paid-woric incentive that is embedded in and well-articulated with preparatory classroom 
activities and on-the-job follow-ups by school personnel. In Des Moines, the integration of a 
paid-work component into an existing school-within-a-school program seems to have led to a 
substantially reduced dropout rate. In fact, the director of the Des Moines demonstration 
credits this component of the program as key to the success achieved during the study period. 
It is important to emphasize, however, that the Des Moines "treatment" was the combination 
of smaller, more personalized classes with a career/vocational service element and the paid- 
work incentive. Similarly, the option of paid-work opportunities on campus for enrolled 
students probably helped the Coleman project achieve a reduced dropout rate relative to the 
comparison site for one of the evaluation periods. In this case as well, however, the paid- 
work strategy was well-integrated into a "family" climate characterized by strong vocational 
preparation. Finally, the promise of a paid job conditioned upon attending a special Saturday 
program for 17 weeks, which featured small classes, career exploration, and job-skills 
development, and maintaining a good attendance record at school during the week led to 
improved absences and lower dropout rates by 1991-92 for students iii Memphis. 

Pay is a tangible reward to students at the high school level, and the responsibilities of 
work can promote the self-perception of being regarded more as an adult than a child. In this 
sense, both the pay and the work constitute the reward for students, and helping youth to 
become adults is in keeping with the larger function of the school. To provide these sorts of 
opportunities, the projects participating in this evaluation utilized both school and community 
resources. In Des Moines, SDDAP grant funds were used to hire school-based work advisors 
who contacted employers to place eligible students in jobs, evaluated the characteristics of 
the jobs that were made available, and foUowed-up with employers and with students to 
ensure that the work experience was rewarding and did not in any way threaten the student's 
continued enrollment. In Coleman, campus work opportunities were supported by funds 
allocated to the school by the Job Training Partnership Act. The Memphis project is 
noteworthy because it drew upon the special talents of a community-based job training and 
development concern with a proven track record of providing assistance to local businesses 
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providing training in job readiness and job-related skills to youth. In fact, the linkage with 
such organizations by schools would seem to benefit all parties. Through such linkages, 
schools arc provided with the job opportunities for students that can be used as an incentive 
for them to remain enrolled, local job developers are provided with job placements that may 
earn cash credits from Federal or State agencies, and both entities should earn the gratitude of 
parents and local business persons. 

Keeping students enrolled does not necessarily ensure that they will receive the types of 
instructional help they need to earn their diplomas, however. Student grade averages did rise 
in Des Moines and in Coleman during 1989-90, and both of these sites did emphasize 
smaller classes, personalized and self-paced instruction, and as-needed counseling services 
for students. In fact, these two demonstrations provided the most extensive arrays of services 
to students of any of the high school sites. Academics, parent involvement, counseling, 
career preparation, attendance monitoring and referrals to other community agencies were all 
part of the Des Moines demonstration, and in Coleman all of these but the parent 
involvement and the attendance monitoring components were provided. The Coleman 
demonstration also provided support services for students, in the form of child-care or parent- 
training classes, as well as other services intended to assist students in meeting their school 
requirements. Clearly these demonstrations suggest that provision of an array of services may 
be necessary to combat problems related to school dropout. 

When multiple services are provided, coordination of services becomes an important 
consideration, and the Coleman project in particular demonstrated the effectiveness of 
several strategies for promoting coordination which involve the selection and organization of 
staff. First, the teachers and counselor at the site were recruited from among experienced 
staff in the seven participating school districts who wanted to work with at-risk students in 
this alternative school setting. All prospective teachers visited the school site and talked with 
the project director and other personnel at the site prior to formally applying for the job. 
Second, these teachers and the counselor were encouraged and sponsored to attend 
workshops on topics related to educating at-risk youth and to visit other schools in the area to 
observe new techniques. Most important, an informal although clearly established support 
system was created to foster the sharing of new information, problems, and insights related to 
student needs. 

As was evident from both the Denver-Discovery and the Memphis Partners projects, the 
failure to assign highly motivated staff and to encourage their continuing development and 
sharing of information related to new approaches is likely to lead to a longer start-up period 
and a less successful academic program generally. In this event also, the services that are 
provided to students are apt to appear as more disjointed. For example, in Denver the efforts 
of the teacher teams may have appeared to students as less connected to the efforts of the 
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conununity outreach specialist, and in Memphis the efforts of the teachers must have seemed 
rather unrelated to the larger-group sessions on building self-esteem. In short, without such 
strategies to ensure coordination of services, it is less likely that the students or the staff of a 
project will describe theirs as a "family effort" 
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Chapter 6. Summary and Implications of Findings 



This report presents the findings from a three-year, intensive examination of the effects 
of 16 dropout prevention and dropout recovery initiatives funded under the School Dropour 
Demonstration Assistance Program (SDDAP) from Fall 1988 to Spring 1991. In Chapter 1 of 
this report, the aims of SDDAP were reviewed, as were the procedures used by the 
evaluation. Chapter 2 reviewed the contexts and general characteristics of the in-depth study 
sites. Chapters 3 through 5, respectively, described these demonstration projects at the 
elementary, middle, and high school levels in greater detail and presented the key findings 
from school records and student and teacher surveys. At the conclusion of each of these 
chapters, summary sections addressed the key policy questions to be informed by this 
evaluation concerning the organizational characteristics of effective dropopt prevention 
programs and the program strategies that are most effective in preventing students from 
dropping out of school or in encouraging dropouts to reenter school. 

We draw together these evaluation findings in this chapter, with references to the 
published literature, in providing a set of guidelines for thinking about and implementing 
dropout-related activities in schools and communities. We begin with a discussion of general 
observations organized with respect to the key evaluation objectives related to organizational 
structure and effective strategies. Next, we use specific observations from the evaluation to 
support five recommendations related to program implementation. Finally, we address two 
tf Dies that are key to continuing efforts to prevent dropout from school: (1) how arc 
innovative program approaches best sustained, and (2) how are innovative program 
approaches best replicated or adapted in new settings. 

At the outset, it is important to stress that the findings from this in-depth evaluation are 
tentative and are based on assessments of student outcomes that are relative in nature. In 
addition, the student samples drawn at each of the sites were not, strictly speaking, matched, 
so even these relative assessments are bound to be confounded by pre-existing differences 
between the groups. Finally, there are the matters of sample attrition and rates of non- 
response (or the unavailability of records), as well as the inherent problems in any set of 
performance measures intended to be useful across a range of projects with different student • 
populations, service goals and priorities, and contexts. For all these reasons, our findings are 
offered only as suggestive for further study by researchers and practitioners. 
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General Observations 



Keeping in mind the cautions that must accompany interpretations of results from this 
evaluation, it seems possible to make eight general observations based on the experiences of 
these 16 SDDAP demonstration projects. The first three of these observations relate to the 
organizational characteristics oi dropout prevention programs, and the five following 
observations relate to effective dropout prevention strategies. 

Observations Related to Organizational Characteristics 

The more complex the organizational structure of a dropout prevention 
initiative (i.e., the greater tendency toward restructuring or non-school-based 
coordination of services), the longer the time period that is likely to be 
required for start-up and the less likely it is there will be evidence of gains for 
students in the short-term. 

Many current educational reform initiatives for students at risk of educational failure 
call for restructuring (or reconstructing) the schools; the messages being delivered by 
educational researchers and practitioners increasingly call for changing the ways in which 
school activities are governed, conducted, and articulated within districts. In Philadelphia, for 
example, Michelle Fine and her colleagues are creating "charters" within school buildings 
that are redefining the roles of teachers and parents and their interactions with students (see, 
for example. Fine, 1992). Similarly, in Chicago, educational reform initiatives have changed 
resource allocation formulas, created new parent-controlled school governance bodies, and 
sought to create school environments that feature closer working relationships among 
teachers, administrators, and parents. Finally, James Comer, Henry Levin, Theodore Sizer, 
and Robert Slavin in various ways have called for altering the status quo in the schools (see, 
for example. Comer, 1988, Levin, 1991, Sizer, 1984, and Slavin, 1990); their particular 
agendas include features ranging from revised decision-making strategies to new attitudes 
regarding curriculum and the processes of instruction. While the current evaluation is without 
comment on the potential of these various systemic approaches, it would appear from our 
findings that the more organizationally complex the dropout prevention intervention, the 
longer the start-up period and the less evidence of increased benefits for students during the 
early years of the effort. 

Complexity in the context of this study refers to (1) restructuring efforts, (2) 
reorganization of school service approaches, and (3) interorganizational coordination of 
services. At the elementary level, one of the study sites (Hannibal) attempted to implement 
an accelerated schools model, which calls for extensive reshaping of decision making and 
curriculum systems. Although the data are sparse, students participating in this demonstration 
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failed to show any greater achievements in attendance or grade averages over a two-year 
period than did a comparison group at a neighboring and "unreformed" school. At the same 
time, school officials report that much progress has been made in creating a new team spirit 
among the school and the community in support of the schools; for example, parent-teacher- 
administrator committees have been established and are working actively to formulate school 
goals and policies and to monitor program effectiveness. 

In the same vein, two high school demonstrations — in Des Moines and Denver- 
Discovery — aimed to reorganize the classroom experiences of students by creating school- 
within-a-school and teacher-team approaches. In these cases, the widely differing indications 
of success for students enrolled in the programs may relate directly to the preparation time 
required for the approaches that were used. In Des Moines, where positive student outcomes 
were quite striking, the school-within-a-school aspect of the program had been established 
for a number of years in each of the participating schools prior to SDDAP funding. In 
contrast, the Denver-Discovery project had come into being with receipt of SDDAP funds 
and used part of the grant period to develop its operational plan for a teacher-team approach 
to working with 9th-grade students. 

Two other demonstrations, in Shreveport and Memphis, attempted to implement 
coordinated service arrangements that prominently involved community-based organizations 
(CBOs) — in one case as the official recipient of the SDDAP grant. The results for these 
projects also suggest that complex interventions may require greater time and greater staff 
effort to produce student benefits. In one case (Shreveport), the efforts of the CBO may not 
have been sufficiently articulated with the school curriculum, such that despite the extra 
assistance in reading and language development provided outside of school, no statistically 
significant gains in grade averages were recorded vor students on in-class assignments. In the 
other case (Memphis), the unilateral efforts of the CBO did achieve relatively higher school 
attendance rates for students; however, the efforts of the CBO and the school to work 
together to assist students academically failed because of a lack of coordination (see the 
discussion below). In both Shreveport and Memphis, however, these demonstration efforts 
were regarded as sufficiently promising for the efforts to be continued beyond termination of 
the SDDAP grants. 

In contrast to the Hannibal, Des Moines, Denver-Discovery, Shreveport, and Memphis 
sites, the Gushing and Portland demonstrations suggest the potential effectiveness in the short 
term for dropout prevention of school-based interventions that add specific services to the 
more regular school routine. Each of these projects succeeded in achieving a pattern of 
positive results for students for one or more years of the evaluation. In Cushing, the project 
component that was included in the in-depth evaluation provided after-school tutoring to help 
students with homework and as-needed counseling related to school and family problems. In 
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Portland, counselors met with students in group sessions and on an individual basis as needed 
to discuss problems at school and problems related to substance abuse in particular. These 
opportunities had been previously arranged with employers by the staff of the demonstration 
project. 

In making these observations, we do not want to appear to be endorsing add-on methods 
over more comprehensive efforts to promote school change. Over the longer term, school 
restructuring, class reorganization, and interagency efforts involving schools may well be the 
most effective approaches. To assess their effectiveness in terms of outcomes for students, 
however, a longer evaluation period (e.g., more than three or four years) is required. 

Coordination of services has the potential to increase the overall amounts of 
services that are available, but such efforts require joint planning and review 
sessions to be successful, and they may require increased funding to maintain 
project efforts. 

In reviewing the nexus of stresses on children and youth that may put them at risk of 
educational failure, Tuma (1989) describes service needs that presently are not being met and 
urges greater pooling of service system resources. She notes, for example, the lack of school- 
based mental health interventions and the problems of coordinating these sorts of programs 
with other systems of care outside the schools. Previous research on interagency coordination 
of services has documented many of these problems in various social service sectors; 
however, this work also has provided evidence that such coordination may lead to improved 
service quality (see, for example, Gans & Horton, 1975; John, 1977; Rossi, Gilmartin, & 
Dayton, 1982). To ensure that greater quality of services is achieved, however, it is necessary 
that the agencies and organizations involved work closely with one another. In addition, 
unless the planning and evaluation of interagency services is itself an interagency 
responsibility, the costs of providing services may increase rather than achieve the hoped-for 
efficiencies in operations that can lead to cost reductions. 

In Los Angeles, coordination of school mental health and classroom services seems to 
have been achieved through joint planning and review processes established by the SDDAP- 
funded KEIP project. In contrast, the projects in Shreveport and Memphis mentioned above 
clearly did not achieve optimal levels of coordination involving school and CBO 
representatives. In both cases, there was little or no articulation in practice of the subject 
matters of instruction in the after-school or weekend demonstrations and the weekday school 
programs — despite the fact that instructional staff in both the Shreveport and Memphis 
demonstrations were teachers in the local school districts. In Memphis particularly, the lack 
of any sort of shared evaluation responsibility between the school district (i.e., the SDDAP 
grantee) and the CBO concerning the academic skills component of the Saturday program 
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resulted in a less than effective instructional component that relied on the large-classroom 
approaches (e.g., chalk-and-talk approaches) typically used throughout the week. Not only 
was the special opportunity for academic instruction represented by the Saturday program not 
used to maximum advantage, but the paid efforts of the teachers were essentially redundant 
with the efforts of teachers during the school week. In this case, neither improved services 
nor reduced costs were achieved by the project. 

Another example of the need for joint planning and review in coordinated service efforts 
to prevent school dropout is provided by the Ethete demonstration. In this setting, where 
Native American adults and white teachers had to work together to achieve optimal results, 
the teachers evidently were displeased with the efforts made by the Native American tutors to 
work with selected students. In not bringing these groups together to discuss the aims of the 
tutoring program and to develop procedures by which such a program might be conducted 
and reviewed jointly, the project missed an important opportunity for positive impacts on 
student performance. As a result, teachers and tutors may well have felt themselves to be in 
some sort of competition, perhaps even to the point that they each were seeking to achieve 
different goals, while the extra resources provided by SDDAP failed to produce incremental 
gains in services for students. 

In contrast to the Memphis and Ethete experiences, the Coleman project seemed to draw 
fully on the creative energies and resources of the seven school districts that joined together 
to establish the demonstration. Similarly, the Los Angeles demonstration appeared successful 
in linking the services provided by mental health specialists and teachers to improve the 
classroom experience for students. In both these cases, study results offer some evidence.that 
demonstration students progressed, and in both cases the levels of training and support for 
staff charged with implementing the demonstrat'von were high. Such support can only come 
in the context of strong administrative endorsement of coordinated programming, and in both 
Coleman and Los Angeles there was evidence of this support. It is important to note, 
however, that, in both demonstrations, costs may not have been reduced as a result of the 
coordinated efforts. New school-based services were added to the existing system and will 
require the continued strong support of administrators to see that they continue to receive 
funding. 

Providing an array of complementary services (i.e., comprehensive services) 
may be the most effective way of meeting the needs of students at risk of school 
failure. 

Each of the demonstration projects that evidenced a pattern of success for students 
during the evaluation period is characterized by a relatively broad repertoire of services. At 
the high school level, for example, the Des Moines and Coleman projects included seven and 
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six distinctive service elements, respectively. In Des Moines, these elements included 
academics, parent involvement, counseling, career preparation, paid-work, attendance 
monitoring, and linkages with other community agencies. The largest numbers of service 
elements offered among the five remaining projects at this level were five elements for the 
Denver-Discovery demonstration and four elements for Memphis. In Denver-Discovery, 
these service elements included academics, parent involvement, counseling, attendance 
monitoring, and linkages with other agencies; in Memphis, they included academics, career 
preparation, attendance monitoring, and linkages. At the elementary level, demonstrations 
Uiat recorded statistically significant evidence of benefits for children also offered broader 
ranges of services (i.e.. Gushing and Los Angeles); however, at the middle school level, the 
relationship between breadth of services and student successes appears somewhat less 
evident. 

In particular, a broader array of services is likely to be necessary in providing help to 
older students because the needs of these students may be more varied, ranging from 
academic instruction to social services. The findings from the 1990-91 survey of all SDDAP 
grantees, which was conducted in Summer 1992, support this conclusion. For example, at the 
elementary and middle school levels, approximately 40% to 45% of the projects reported 
providing comprehensive services, as compared to more than 60% of the projects at the high 
school level. Equally important, projects that offered a comprehensive array of services were 
reportedly the most likely to be fully utilized. As noted at the outset of Chapter 5, 85% of 
projects offering comprehensive services to students in grades 9-12 reported operating at or 
beyond 91% of capacity, as compared to only 62% of projects offering a more limited array 
of services. 

Observations Related to Effective Strategies 

Counseling services and adult advocacy for students are key elements of any 
particular dropout prevention initiative. 

In the context of dropout prevention, counseling services typically include listening to 
student problems and making attempts to help students resolve these problems. In contrast, 
advocacy for students by adults at school (e.g., teachers, administrators, counselors) refers to 
active adult involvement in representing students' concerns to others for the purpose of 
changing conditions that appear to negatively affect youth. This finding, concerning the 
salience of these types of services, is related to the developmental needs of youth as well as 
the differing (and increasing) demands of our public education system. 

Early in their school careers, students at risk may especially need relationships on 
campus in which they feel they can speak their minds and talk about their worries and 
problems. For example, LeCompte and Dworkiii (1991) point out that students at risk 
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currently do not receive sufficient counseling and call for an increase in counseling services 
that help students meet their "pressing socio-emotional and economic needs" so that they can 
have the peace of mind to achieve well in school. In addition, Morse (1988) and Molnar, 
Rath, and Klein (1990) have shown that inadequate counseling and support services may 
especially hurt young people whose families move frequently or are homeless, and they 
conclude that these young people need counselors who recognize and are supportive of their 
needs. 

The transition from home care or pre-school activity to the elementary classroom for 
some students, for example, may also create stresses and lead to anxieties, with the result that 
these students may require reassurance and reaffirmation of their self-worth to deal with 
these stresses successfully. Increasingly, as elementary classrooms become more crowded, 
diverse, and more demanding, teachers at these levels may find themselves with insufficient 
time to devote to all students. As a result, we see projects such as those in Gushing, 
Shreveport, and Los Angeles that offer counselor time or the time of adult volunteer mentors 
to students as part of a concerted effort to promote school engagement. In addition to our 
finding some positive results for these project strategies, McPartland and Nettles (1991) and 
Maeroff (1990) have shown that such service strategies may have positive influences on 
student grades, attendance, and attitudes towards school. 

At the middle school level also, where another transition — from the elementary to the 
secondary grades — may create its own brand of stress and anxiety for teenage students, we 
see informal counseling taking place on Portland's George Middle School campus, with 
more formal counseling sessions provided to students during class periods at the school. 
Similarly, in the Denver-Lake project in 1989-90, which also served middle school students, 
the Gold Team counselor was available to listen to students' problems, and she also worked 
closely with teachers in handling disciplinary matters. During the middle school years, 
students are particularly vulnerable to peer pressures to engage in destructive, risk-taking 
behaviors (e.g., Lewis & Lewis, 1984), and the presence of adult friends who offer guidance 
without recrimination or accusation is likely to help in counterbalancing these pressures. The 
increasing importance of peer influences during this stage of development is also reflected in 
the Model School Adjustment Program's reliance on peer tutors to provide supplemental 
instructional assistance, although the results from the MSAP replications included in this 
evaluation were rather disappointing in terms of the effects on the students receiving this 
assistance. 

As students grow older into their teen years, the prevalence of self -destructive and 
suicidal behaviors increases (e.g., Huffine, 1989), while at the same time youth are likely to 
become increasingly aware of the need to prepare for their futures (e.g., further schooling or 
work). Stanton-Salazar (1990) notes that a network of supportive adults may increase young 
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people's achievement motivation and improve their awareness of educational and 
occupational opportunities beyond high school; however, during this transition period to 
adulthood, many students and their teachers may find that this sort of network must be 
established without parents. As school officials begin directiy to work in more supportive 
ways in helping older students with personal problems and considerations for the future, they 
may be more likely to regard problems on campus as a function of student-school 
interactions — thus opening the way for student advocacy roles. We find, for example, that in 
both the Portiand and Denver-Lake middle schools and in the Memphis, Coleman, and 
Denver-Discovery secondary programs, counselors and outreach specialists woriced with 
teachers to clarify possible misunderstandings about assignments and worked with parents, 
other community service agencies, and employers to build support systems outside of school. 
In Coleman particularly, school counselors and oftentimes the teachers themselves served in 
place of parents for students who were emancipated minors, assisting them in interacting 
with various service agencies. The efforts of these demonstrations may have overcome the 
superficiality and short-term duration of many adult advocacy programs, which typically 
doom them to failure (e.g., McPartland & Netties, 1991; Freedman, 1988 and 1991). 

At the elementary level, providing after-school tutoring and enrichment that 
are directly related to in-class assignments and having in-dass adult friends 
(e.g., trained volunteers or helpers) may be effective approaches. 

The results from the Gushing demonstration begin to suggest that providing students 
with individualized assistance in completing or mastering classroom assignments may have 
beneficial effects on recorded school performance. This accords well with Kiesling's (1984) 
findings that show pxjsitive relationships between individualized instruction and the reading 
performance (especially comprehension) of elementary students. In the case of the Gushing 
project, tutoring during the after-school hours likely also served to (1) reinforce positive 
attitudes about school and (2) provide care for children during the time many parents were 
completing their work days. In Shreveport also, after-school classes in reading and language 
skills coupled with enrichment activities contributed to students' perceptions of 
improvements in their class work and near-significant findings related to improved grades. 
Finally, by bringing adult friends into the classroom, to listen to individual students and to 
work with them on tasks and be with them in play, the Los Angeles K-1 demonstration 
produced measurable improvements in students' relations with peers and their language 
development during the project's period of initial and perhaps most intensive service with 
these students. As Bronfenbrenner (1989) notes, children need at least one adult who is 
"crazy" about them in order to develop intellectually, emotionally, socially, and morally. 
Having adult friends available to engage especially shy or boisterous K-1 students in 
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constructive activity that is mutually satisfying may be one way to promote the closeness that 
is essential to furthering the psychological development of these young children. 

At the middle level, team teaching strategies, flexible scheduling, 
heterogeneous grouping of students, and provision of as-needed counseling 
assistance are useful strategies. 

The Denver-Lake demonstration used team teaching together with flexible scheduling 
and heterogeneous grouping of students to create a perceptibly warmer and more secure 
climate for learning. The results were fewer absences and suspensions for students during 
the 1989-90 school year— when these characteristics of the project were most evident At 
Portiand's George Middle School, the demonstration provided as-needed counseling and 
advocacy for students on campus in both formal and informal settings and achieved 
improved attendance rates and grade averages for project participants relative to comparison 
students. To an extent, the accomplishments of these various strategies may be attributed to 
the changing of the typical middle school environment, from the junior high school, where 
students move throughout the day between various teachers and see counselors only when 
discipline problems arise or when it is time to schedule classes, to a setting where new 
surroundings and greater personal freedom are mediated by organizational and staffing 
arrangements tiiat preserve a sense of adult caring and responsibility for individual students. 

Team teaching, particularly interdisciplinary teaming, has been a focus of interest of the 
National Middle School Association and led to the recently released issue of Research in 
Middle Level Education entitied "Interdisciplinary Team Organization" (National Middle 
School Association, 1992). In that volume, evidence is provided that teaming reduces 
(individual) teacher isolation and increases teacher professionalism, although teacher teams 
may find themselves somewhat more isolated from school staff persons more generally. In 
addition, Mac Iver (1990), Robinson (1991), and McPartland (1992) cite evidence tiiat attests 
to the effectiveness of interdisciplinary teams in creating more positive school climates. 
There is less agreement in tiie literature, howf ver, about the effectiveness of either flexible 
scheduling or heterogeneous grouping of students. The majority of studies on flexible (block 
or modular) scheduling were carried out in the mid-1960s, and the findings from tiiese 
studies supported claims that such systems were difficult to manage administratively. In 
terms of student outcomes, selected studies did show that flexible scheduling resulted in 
student reports of more time with teachers and student perceptions of greater instructional 
benefits (e.g., Braddock,.1967). On the matter of heterogeneous grouping the results are also 
mixed and are reviewed below in discussions of specific principles for implementation. 
There is some evidence, however, Uiat such grouping promotes the learning of lower ability 
stiidents (e.g., Veldman & Sanford, 1984) and reduces the salience of negative peer 
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influences (e.g., Feldman, Caplinger, & Wodarski, 1983; Hallinan & Sorensen, 1985; and 
lanni, 1989). 

At the secondary level, paid-work, embedded in activities Chat prepare and 
monitor students' on-the-job experiences, appears to be a critical component to 
keeping students in school. 

Stinchcombe (1964) has argued that the rebellious behavior and psychological 
alienation of impoverished adolescents is partly attributable to their perceptions that social 
conformity and academic achievement are not clearly linked to future job opportunities. 
More recently, in a review of research on successful dropout prevention programs, Hamilton 
(1987) finds that these programs commonly have strong vocational components, and the 
Grant Foundation Commission on Work, Family, and Citizenship (1988) has called for 
"bridging the gap" between school and work by showing students the relation between 
education and job market success. Perhaps in some sort of response to these fmdings and 
remonstrations, the 1990-91 survey of all 89 SDDAP grantees showed that 74% of grade 9- 
12 projects reported offering career awareness/job counseling services, 39% reported 
offering vocational training, 31% reported offering work study/job internships, and 36% of 
these projects reported offering job placement services to students. 

In the in-depth portion of the evaluation, the SDDAP high school demonstrations that 
recorded discernible patterns of accomplishments, including lower dropout rates relative to 
comparison students, all utilized paid-work experience as a key incentive and means of 
linking the value of schooling tangibly to the life context. In each of these programs, 
however, the paid-work incentive was preceded by career exploration and vocational 
preparation activity and followed by adult monitoring of the on-the-job experience to ensure 
there was no interference of job activity with students' interest and ability to complete school 
assignments. In Des Moines and Coleman, paid-work opportunities were available to 
students on or off campus as long as they remained enrolled; in Memphis, a job was assured 
if students persisted in the 17-week academic and vocational skills development program and 
remained enrolled in good standing at their "home" schools. Importantly, along with the 
work, these three projects also offered students access to advocate-advisor-mentors, who 
assisted them with school-related problems and in their interactions with community 
agencies and employers. These individuals were proactive in their approaches, seeking out 
students to check on status rather than simply waiting to be contacted. 
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!n programs where dropout recovery is an emphasis, flexible scheduling of 
school work assists students who need to work or meet personal commitments 
during regular school hours. 

The Carbondale demonstration evidenced the benefits for some current and former 
students of flexible scheduling of instrucdonal periods. Precisely because some students need 
to work during the day or care for their children, Operatiori Rebound allowed students to 
schedule late afternoon or early evening meetings with instn<ctors. The combination of this 
sort of flexible scheduling with paid-work opportunities created through school-business 
partnerships that arc linked to continuation in school would seem a most promising strategy 
for combating school dropout among older students. For those who have already dropped 
out, however, the combination of these services may prove less useful. For example, Hahn 
and Danzberger (1987) conclude that research to date shows that offering paid-work to 
already dropped-out students is not an effective lure for ihese students to re-enroll. These 
researchers suggest that dropouts may have so many other problems (e.g., lags in grade level) 
that they often do not want to return to school even if they are guaranteed a job. 

Recommendations for Implementation 
Based on Specific Observations 

The findings and observations reviewed in the previous three chapters of this report lead 
us to make five recommendations related to the implementation of dropout prevention 
programs. Since these recommendations are based solely on the findings of this evaluation, 
they must be regarded only as suggestive. The fact that they confuin findings reported by 
others does, however, lend credence to their significance for future efforts. We present our 
recommendations here, drawing on both the findings from this evaluation and other findings 
reported in the literature to provide the necessary rationales. 

It is important to recognize, as has been stressed at the outset of this chapter, that both 
the general observations from this evaluation and our recommendations are based on a 
patchwork of results. Specifically, we have attempted to generalize themes from individual 
results at particular sites; it is not the case that all of these themes are based on similar 
flndings replicated across all the demonstrations. In addition, our assessment of "project 
success" has been based, in the main, on the relative assessment of student performance; 
when we report that fewer absences or higher grades have been achieved, for example, these 
judgments are made relative to a comparison group and may not at all reflect absolute 
improvements in the performance of individual students. For these reasons, there is a fragility 
in the results that must be emphasized. What is needed is further careful testing of the 
generalizations and recommendations presented in this report, preferably in studies that 
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utilize experimental designs (i.e., random assignment). While they arc demanding, these sorts 
of designs arc necessary to determining the efficacy of specific instructional strategies. 

Recommendation 1: Put the services in rather than pull the students out. 

In the 1990-91 survey of SDDAP grantee efforts, 50% of K-8 projects, 14% of grade 
9-12 projects, and 60% of multilevel projects (i.e., projects serving students at various grade 
levels) were found to be using pull-out instructional strategies. (These strategies typically 
include removing students from their regularly scheduled classes to permit their participation 
in special instructional programs.) Overall, from 50% to 75% of the projects using pull-out 
strategies at each of these grade levels reported them to be of high importance. 

The most likely rationale for taking this instructional approach is that it is much less 
efficient to build into regular classrooms the types of special activities, often individualized 
or one-to-one activities, that (only) specific students may require. In addition to possibly 
allowing several students at once to receive the special instruction-related service from a 
single teacher, removing students with special needs to another classroom for a period, for 
example, would free the teacher in the regular classroom to continue with scheduled 
assignments. When large numbers of the students being taught in classrooms are in need of 
special instructional services, however, pulling out only selected students for extra help is 
likely to be much less effective than reorganizing the general classroom instructional 
approach. With such reorganization, which may include strategies such as continuous 
progress models and cooperative learning, pull-out programs are likely to be less useful and 
may actually introduce unnecessary costs (e.g., Slavin, 1987). 

In this in-depth evaluation, the dropout demonstration projects that utilized pull-out 
strategies achieved less convincing patterns of desired student outcomes than did other 
projects. The Broward County and San Antonio projects at the middle school level and the 
Ethete demonstration at the high school level all used these types of strategies with few 
positive results as assessed in this evaluation. In each case, one or two students were pulled 
from their regularly scheduled classes for all or part of an instructional period: in Browanl 
County, students worked with peer tutors and sometimes with a counselor during puU-out 
sessions; in San Antonio, they worked with special teachers; and in Ethete, they met with 
Native American tutors. The San Antonio project director did terminate the pull-out aspect 
of that demonstration beginning in 1991-92, citing dissatisfaction with the results being 
achieved. Much of the impetus for this change resulted from evidence that larger numbers 
of students required special instructional services than were being served using a pull-out 
approach. At the same time, this director expressed concern about the stigma for students that 
might be associated with being pulled out of their regular classes and the negative effect of 
this perception on their in-class performance. 
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The issue of stigma has been a part of previous examinations and critiques of pull-out 
programs for use with students at risk. Glass and Smith (1977), for example, note the likely 
harmful effects on students' self images of being "taken away" for special instruction. In 
addition, Natriello, McDill, and Pallas (1990) review the criticisms of this "[most] 
controversial aspect of Title I/Chapter I," noting that critics have charged "these programs 
stigmatize or 'label' compensatory education students as inferior in the eyes of both teachers 
and student peers" (p.77). 

In San Antonio, students may well have perceived being pulled out of class for special 
instruction as a stigma; however, in Ethete quite the opposite may have been the case. In that 
site, the students were Native American, the tutors were Native American, and the teachers 
were white. Rather than feeling stigmatized by being pulled from the regular class, Ethete 
students may have regarded their interactions with adult Native American tutors either as 
status enhancing or as important opportunities for reaffirming cultural identity, or both. 
Finally, in Broward County, stigma may actually have arisen from two sources. First, as with 
any pull-out program, MSAP students may have felt themselves singled out for special help 
"with their classmates watching." Second, the fact that the tutors who came to fetch them 
from their regular classes were their peers may have increased the sense of embarrassment 
for MSAP students at being singled out. In reality, this second factor may be more complex, 
since these peers were older students with good understanding of the subject matters and may 
have inadvertently caused the MSAP students at the replication sites to feci somewhat like 
younger brothers or sisters. 

In addition to stigma, pull-out programs may experience problems in coordinating 
activities with regular classroom teachers. For example, Natriello et al. (1990) point out that 
critics have taken issue with these programs for several other reasons besides stigma: 

1. There often is a lack of coordination between instruction in the regular and pull-out 
classes, with teachers rarely and pooriy communicating. The result is that pooriy 
achieving students are burdened with having to reconcile different types of instruction 

2. Pull-out programs often "supplant" rather than "supplement" instructional time in 
basic skills, since the more compensatory programs in which a student is involved, 
the less instructional time she or he receives 

3. These programs lead to a diminution of the responsibility felt by regular classroom 
teachers for the academic welfare of disadvantaged students (p.77) 

Aspects of each of these problems are evident in the three SDDAP pull-out demonstrations 
and may help to account for their appaient lack of consistent success. 

In San Antonio, for example, one problem with the pull-out tutorials was that students 
were not able to earn credits toward graduation in these settings. Not only was instructional 
time being supplanted, but students were fmding that being pulled out really did not help 
them advance toward a diploma. Students in the Broward County program did not have to 
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worry about credits during their pull-out periods, since these students and their peer tutors 
worked on regular classroom assignments during the pull-out period. There remains, 
however, the issue of what these Broward County students were missing from their regular 
classes on the days they were pulled to work on assignments with their tutors. In Ethete, what 
teachers thought the pull-out students were missing when they were excused from classes to 
meet with their Native American tutors led only to resentment. The teachers at this site 
reported seeing little connection between what the tutors were doing and what they were 
seeking to accomplish, and tiiey worried that tiie students would fall farther behind their 
classmates in completing various assignments. 

Recommendation 2: Deliver the services without calling attention to the fact 

that special services are being provided. 

Just as stigma may be associated witii periodic pull-outs from regular classes, stigma 
may accompany the perception that certain students in regular classrooms are receiving 
something other than the regular course of instruction. la either of these cases, increasing the 
sense of community (e.g., shared purpose, mutual respect) among school staff and students 
may be die preferred long-term solution. In the short term, however, unobtrusive ways must 
be found to integrate tailored instructional programs within typical classroom settings as well 
as in alternative educational programs. 

LeCompte and Dworkin (1991), for example, have stated that dropout prevention 
programs tiiat single out students at risk are ineffective. They write that ". . .most of tiiese 
students do not want to be singled out, isolated, and stigmatized" (p.209). In referring to 
alternative schools, these autiiors further note that "While many of them don't want to return 
to tiieir home schools, it is because their experiences in their home schools were so 
unpleasant, not because tiiey want to be segregated with other dropouts" (pp. 209-210). The 
results from die exit interviews we conducted in Coleman— an alternative school project and 
one of die more consistentiy successful of the SDDAP demonstrations panicipating in the in- 
depth evaluation— reflect this attitude, with students reporting among die salient reasons for 
dieir leaving die lack of diversity of die student body at die alternative school. 

Tracking is one of die ways schools may attempt to provide differential instructional 
services to students at different ability levels on die same campus; however, the debate over 
tracking has bodi its advocates and harsh critics. Braddock (1990) concludes that the effects 
of ability grouping are especially negative for African American, Hispanic American, and 
Native American students in the lower tracks, and Oakes (1989) argues diat tracking in 
secondary schools stigmatizes tracked students and lowers teachers' expectations for them. 
On die odier hand, Kulik and Kulik (1.982), based on a meta-analysis of research flndings, 
have suggested diat ability grouping does not adversely impact lower-track students and has 
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a slightly positive impact on the academic performance of higher-track students. Slavin 
(1990) also reviewed various study results and concluded that ability grouping has no effect 
on achievement for either lower- or higher-track students. The issue of stigma remains, 
however, a serious criticism for any attempt to meet learner needs that involves segregating 
students — whether or not achievement test scores or others of the "standard" indicators of 
performance register improvements. Of those SDDAP demonstrations participating in the in- 
depth study that recorded gains for students relative to the comparison groups, all found ways 
to avoid these sorts of issues either by (1) avoiding entirely ability grouping as a prerequisite 
for receiving special services or (2) increasing the perceived benefits to students of receiving 
the special services to such an extent that the negative perceptions of grouping were 
effectively canceled. 

In Los Angeles, adult mentors in the classroom appeared to be regarded as special by 
all the students, and in Gushing and Portland, respectively, the counselors were there for all 
students. The Denver-Lake project randomly assigned students to the Gold Team and then 
altered the regular instructional program for all the Team members. When certain Team 
members were provided bilingual instruction to build their English skills, for example, other 
members received bilingual instruction to further develop their verbal skills in Spanish. 
Finally, although applications were required, the Memphis demonstration was available to 
all students who wished to participate for the 17- week period. Only the Des Moines and 
Coleman projects, arguably the two projects evidencing the most consistent records of 
accomplishment relative to dropout prevention, did engage in clustering nnost at-risk 
students. In these cases, however, attempts were made to create alternative school environ- 
ments for students that would be perceived as providing opportunities for learning and work 
that were simply unavailable in the regular school situation. 

In Des Moines, the School-Within-A-School and New Horizons students were selected 
for the program by referrals based on their grades, ITBS scores, and attendance patterns. In 
Coleman, the referral process was based less specifically on grades or test scores; however, 
these arc included in an expansive list of at-risk dimensions developed by the Texas 
Education Agency for identification of students in need. As noted above, however, both of 
these projects sought to avoid the stigma of tracking by creating alternative programs that 
provided distinctive advantages to students. In Des Moines, these advantages included paid- 
work opportunities, and in Coleman these advantages included paid-work plus the 
opportunity to rely on project staff to help with the full range of personal problems. 
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Recommendation 3: Deliver the services within a supportive climate that 

includes adults as student advocates. 

The reliance of Coleman students on their CAP teachers for help with personal 
problems evidences the sort of close, caring relationships that (also) appear to be necessary 
for achieving successful rRsults in terms of school performance. In Coleman, Portland, 
Denver-Lake, Memphis, and Denver-Discovery, developing especially close relationships 
among staff and students was a priority, and in Denver-Lake and Coleman these efforts 
resulted in documented student perceptions about the improved quality of their school 
climate. As noted previously, the counselors in Portland and the counselor and outreach 
specialist in Denver-Discovery also served as student advocates, interceding on behalf of 
students with teachers and sometimes with their parents. This theme of care, concern, and 
advocacy also runs consistently through the literature on working with students at risk. 

Bryk and Driscoll (1988) noted the importance of building supportive, caring climates 
around students at risk, and Schon- and Schon- (1989) reported that most effective programs 
for disadvantaged students describe their staffs as skilled and highly committed, serving as 
roie models for both parents and children emulating care and concern. As if to sum up 
findings in this area, Natriello et al. (1990) cited positive relations in school as one of four 
elements of effective programs for secondary students at risk. Specifically, these authors 
pointed out that access to counseling staff may help students at risk form positive ties and 
feel that someone at the school cares about them. Indeed, supportive and caring school 
climates that embody a sense of community built on mutual trust between school staff and 
students may be key to avoiding any stigma associated with tracking or the special grouping 
of students into alternative school arrangements. 

Recommendation 4: Provide students with substantive incentives to 

participate, or to receive the services. 

This in-depth evaluation of SDDAP demonstrations seems to provide rather consistent 
evidence of the salience of student incentives for achieving outcomes related to dropout 
prevention. No matter whether students were required to apply for dropout prevention 
programs (e.g., as was the case in Memphis) or simply were selected by school officials to 
receive the services (e.g., as in Des Moines), substantive incentives appeared to be effective. 
For example, at the elementary level, students in Cushing received help in seeing that their 
assigned homework was completed correctly and in timely fashion as a result of participating 
in the after-school tutoring program. This program, together with the help provided to 
students in getting their homework done, likely resulted in the documented perception that 
they were doing better on class assignments. At the middle school level, Portland counselors 
served students as adujt friends with whom they could share any problems and ask any 
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questions at any time. The availability of these adult friends likely resulted in improved rates 
of absences and higher grades for project participants relative to comparison students. 
Finally, at the secondary level, the Des Moines, Memphis, and Coleman projects provided 
paid-woric and vocational training opportunities for students who remained enrolled or 
completed their project commitments in good standing. In Des Moines this strategy may have 
helped to achieve higher grade averages for students and lower dropout rates relative to 
comparison students; in Memphis it consistently led to improved attendance records at 
students' home schools (relative to comparison student records); and in Coleman, paid-work 
opportunities were one of the elements that likely resulted in higher grades and evidence of 
reduced dropout of program participants relative to comparison students during 1989-90. 

These types of incentives, as well as less substantive but no less tangible incentives such 
as certificates, designer watches, and money, have also been found to be effective in a range 
of other studies. Hahn and Danzberger (1987), for example, describe various incentive 
programs for elementary and secondary students, including those offering material incentives 
(e.g., watches) and those offering emotional supports (e.g., mentors), and argue that offering 
such incentives is an important element in motivating young people to stay in school. 
Similarly, Mac Iver (1991), in reporting on Year One of the Incentives for Improvement 
Program, also presents evidence of the positive effects of rewards on student performance 
and motivation to learn. Finally, Zeldin, Rubenstein, Bogart, Tashjian, and McCoUum (1991) 
write that many underachieving students need special incentives just to stay in school. In fact, 
incentives have, for the most part, been found wanting in evaluation and research studies 
only when they have involved deferred rewards. For example, Gottfredson (1988) criticizes 
such long-term rewards (e.g., college scholarships upon graduation) on the grounds that they 
are too far away to influence the behaviors of students most in need of assistance. Similariy, 
Natriello et al. (1990) criticize longer-term reward programs and suggest that incentive 
programs that give incremental pay-offs to a students' college scholarship accounts, for 
example, or tie jobs to short-term improvements in students' school records and behaviors on 
campus are likely to be most effective. 

This emphasis on perceptible, extrinsic rewards for students should not be interpreted as 
a criticism of efforts to socialize youth to respond to internal motivations to remain enrolled 
or to perform well in school. Rather, in the absence of such internal motivations, substantive 
incentives provided by school officials may provide the initial encouragement necessary for 
many students to begin to understand and to internalize school-related values. School 
personnel involved in dropout prevention or recovery efforts should not assume that the 
students they serve already possess these internal motivations or are likely to develop them 
upon perceiving that special services are being provided. At the same time, incentives must 
be integrated with the instructional program and, in the cases of secondary students in 
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particular, preparation and follow-up activities are likely to be needed to ensure the desired 
results of such strategies as providing paid-work experience. 

Recommendation 5: Carefully select, train, and support the staff persons 

providing the services. 

LeCompte and Dworkin (1991) review factors that affect teacher burnout, and call for 
more practical in-service trainin/j of classroom teachers. Specifically, they recommend 
greater focus on relating the content of training to actual classroom situations, rather than 
emphasizing theoretical distinctions or focusing merely on the motivation of staff. In the 
1990-91 survey of the SDDAP grantees, 85% of the grantees at the K-8 level reported 
having some staff with special training; 26% reported these staff had received training prior 
to working on the project, and 67% reported staff had received training while working on the 
project. At the same time, almost 30% of these grantees reported that the shortage of trained 
staff was their greatest obstacle to providing services to students. From the in-depth study of 
these SDDAP demonstrations, we are convinced that successful dropout prevention efforts 
are ones that select staff carefully and provide initial staff orientation that is more than 
description of the project aims and staff roles. Following orientation, these projects continue 
to provide skill-building opportunities, counseling, problem-solving sessions, and 
motivational aids to staff to maintain necessary focus on key goals and necessary interests 
and abilities in providing services. These types of staff supports appear particularly important 
when the prevention program involves the combination of various services (e.g., when the 
program is comprehensive), and they seem to become critical when the program represents a 
new direction from the more traditional, regular school program. 

In Coleman, for example, only teachers with experience in working with students at risk 
were asked to apply for positions. Upon application, these teachers were required to visit the 
alternative school and to talk with the project director and other staff already selected about 
the work environment and the project puipose. Only after teacher-candidates were debriefed 
following these visits and expressed themselves still interested in positions were they 
advanced to the actual selection pool. In addition, orientation in Coleman was really a group 
affair, with teachers sharing their ideas as they welcomed new staff into their CAP family. 
Further, all Coleman staff were actively encouraged to attend skill-building workshops 
related to methods and curricula for students at risk, and periodic meetings of staff 
throughout the year were specifically arranged to promote the sharing of problems and group 
brainstorming about possible solutions. 

In contrast, in Denver-Lake and Denver-Discovery, where teacher teams were relied 
upon to provide instructional services and periodic counseling to students, the selection of 
teachers and their orientation, training, and continuing support were aspects of the 
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demonstrations that were less well integrated into the project design. For 1989-90, Denver- 
Lake did manage to create a distinctive climate for students that produced reductions in 
student absences and suspensions; by 1990-91, however, team members had left the project 
and the team counselor position had been eliminated. Perhaps as a result of these events, 
student accomplishments for that year and any continuation of the benefits identified in 
1989-90 were not evident. In Denver-Discovery, teachers in 1990-91 showed signs of 
having accommodated themselves to a team approach, but several team members expressed 
their hopes for transfers out of the program at the end of the school year. The building of 
bridges for the program with other teachers and the central administration, which might have 
been accomplished in part by the efforts of the teacher teams, seemed stalled. (In fact, the 
perceived isolation of these teacher teams may have contributed both to their lack of interest 
in continuing as team members and their lack of effectiveness in connecting to teachers in the 
regular school program; see Mills, Powell, & PoUak, 1992, for further description of the 
possible causes and effects of the isolation perceived by teacher team members.) Finally, in 
Memphis, where the teachers seemed to make ineffective use of small class sizes of 
motivated students, there was no monitoring of teacher efforts (as noted earlier) and no 
attempt was made to help teachers exploit the instructional advantages that were provided. 

At various other demonstration sites, such as Broward County, Aiken, and Carbondale, 
staff training may be relatively less important because the service requirements for teachers 
were not that different from the regular classroom. At each of these sites, for example, 
training after an initial orientation was neither a priority nor was it obvious that it should be; 
these demonstrations almost operated independently of the specific staff persons who were 
assigned to them. 

Sustaining Dropout Prevention Programs 

Our study of factors related to sustaining innovative approaches was limited to visits to 
selected projects from among the in-depth study sites that had evidenced patterns of success 
in working with students at risk sometime during the SDDAP grant period: Des Moines, 
Coleman, and Denver-Lake. Our aim in these visits was to identify factors that had 
contributed to maintaining the programs during the grant period and in the year following 
that period or, in the case of Denver-Lake, to leam about the conditions that contributed to 
the dissipation and eventual termination of project activities. 

The project in Des Moines focused on providing mentored work experience 
opportunities for high school students who were participants in a school-within-a-school 
(SWS) program. Work experience counselors did provide some instruction in career 
exploration activity as part of the school-within-a-school program, but they did not assist on 
a daily basis in the pi; ning of the non- vocational SWS curriculum. To keep the project 
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going, program supervisors had to ensure that work experience staff remained committed to 
their tasks and energetir in their efforts to work with employers and to follow-up on students 
placed in various jobs. This they accomplished in large part by creating a sense of team spirit 
among the work experience counselors and the SWS teachers. 

The Coleman and Denver-Lake projects also sought to achieve their aims through the 
development of teams that were able to woric closely together and build a sense of "family" 
with the students served. (The Coleman project worked with high school and older youth, 
while the Denver-Lake demonstration was carried out with youth of middle school age.) In 
each of these projects, the nurturing of the teams was critical; to sustain their activities, each 
project had to ensure that team members continually "bought into" the goals and tasks 
associated with the project and remained fresh in carrying out their work. In both cases as 
well, it was critical that nothing within or outside the school sites (e.g., at the district level) 
should work to the detriment of the teams. 

Reinforcing Staff Commitments to the Program (Team Spirit) 

Staff of the Des Moines, Coleman, and Denver-Lake projects developed a camaraderie 
that helped encourage and sustain their commitments to the projects. Whatever the energy 
levels or conflicting concerns of individual staff members on particular days, the sense of 
being part of a team was likely to have helped in focusing attention on the shared aims of the 
project. 

The staff persons who worked at these three sites came from a variety of backgrounds, 
with varying types and levels of experiences in teaching and working with students. They all 
volunteered to be part of the programs, however, and knew from the outset they were 
participating in special efforts. During the first two years of the SDDAP grant period at each 
of these sites, considerable attention was devoted to building team spirit. 

The most common strategy used in building staff teams was to provide the staff with 
time— the time to talk with one another, to plan activities and compare notes. In Coleman and 
Denver-Lake, for example, SDDAP funds were used to provide joint planning sessions 
during the school day; in Des Moines, work experience coordinators had the time to meet and 
review class plans and students' progress at the various work sites with SWS teachers and 
with their counterparts at the other high schools. 

A second strategy used in team-building was to instill a sense of joint ownership in the 
projects. Given that the staff persons were involved in special dropout prevention efforts, 
they were enlisted as planners and evaluators of the program. In Des Moines, for example, 
staff had free rein to locate jobs and arrange for work-site mentors for students. In Coleman, 
teachers were encouraged to develop special projects for students on campus (e.g., helping to 
construct a GED preparation center). And in Denver-Lake, the teacher-counselor team was 
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responsible not only for planning and scheduling the weekly curriculum (including the 
instruction in elective courses), but for all student discipline and organization of student 
field-trips as well. 

A third strategy was to attend to the performance of the team as a team. Specifically, 
project managers or principals monitored how well their teams were functioning, were 
careful to praise and characterize project successes as products of team efforts, and did not 
hesitate to make staffing changes when problems arose. Teachers who "didn't fit" were 
reassigned, and new teachers or counselors were typically brought into the projects after 
consultation with resident staff. In Coleman, for example, new teacher candidates were 
brought in first as observers and then were asked to meet with current teachers to discuss 
project activities.. Only following these activities and discussions involving resident teachers 
and the principal would candidates be selected for work at the site. When this sort of team 
management and consultation faltered, as it did at the Denver-Lake site following the 1989- 
90 school year, the sense of team spirit was severely diminished. 
Keeping Staff Fresh in Pursuit of Dropout Prevention Goals (Challenge) 

At these three sites, staff clearly were challenged to make a difference for students at 
risk. The New Horizons program in Des Moines, which had assisted students generally for 
several years, was now to be put to the test in woricing with the SWS population. Similarly, 
Coleman's CAP program was to be the last resort for the at-risk or endangered youth of 
seven other school districts. Finally, the Denver-Lake project was to remake the middle 
school experience in the city at a time when 6th-graders, for the first time, were to be 
assigned to middle school campuses. This sense of challenge also characterized project 
efforts to keep staff motivated in their work with students. 

One strategy that was used to challenge staff involved providing release-time (or travel 
costs) for training and staff development sessions. The Coleman site, in particular, 
' encouraged staff to attend workshops on various topics related to working with youth at risk 
(e.g., conflict resolution). Another strategy was to publicize program achievements, using 
these achievements to build community support and expectations for the projects. In Des 
Moines, for example, press releases featuring student-participants in the work-experience 
program and meetings with business leaders were credited with building administrative 
support for the project within the district. Such support, in the form of local funding for the 
program, would be likely also to provide an impetus for staff to try new things, such as 
seeking out new employer relationships. A third strategy was to introduce new staff to the 
projects by rotating existing stc^into other positions. In both Des Moines and Coleman, new 
staff have come to the projects as other staff have transferred to new assignments; in 
Coleman, the assistant principal for the project became its principal in 1992-93, bringing 
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with him several new ideas for activities. At this site as well, one of the original teachers will 
shortly assunic responsibilities as dean of students, allowing a new teacher to join the 
alternative high school program. 

In contrast to the use of staff reassignments to keep a prograin fresh in its apptt)ach, 
Denver-Lake provided an example of how unplanned staffing changes and turnover may 
create serious problems for attempts at school change. Prior to the start of the second year of 
the SDDAP grant period at the site, the author of the school's SDDAP application was 
unexpectedly transferred. The teacher team assembled for the first year included one teacher 
who was assigned by the district and two staff persons who were not assigned until later in 
the year, by the end of that year, three team members had either been reassigned or had opted 
for new positions. Following the second project year (1989-90), three teachers also requested 
reassignment, one of whom had been the teacher-leader of the team to that point. In selecting 
new staff for 1990-91, one of the three new hires was assigned to the team by the district 

Establishing Connections to Existing Programs (Bridging) 

The Des Moines work-experience program was married to the existing school-vrithin-a- 
school program by SDDAP project designers. Each program was aheady established within 
the district's schools, and the combined service approach represented a blending of two 
complementary activities. With this base of acceptance, the pixjject had only to prove itself 
with the business community, which was only too eager to lend support while at the same 
time attracting applicants for entry-level jobs. At this site, the absence of difficulty in 
establishing connections to existing programs allowed staff a free hand to carry out their 
activities. 

In Coleman, the demonstration project was created by a consortium of several school 
districts, the superintendents of which would serve as the Board for the newly created 
alternative school. From the outset, CAP was to serve the needs of the founding districts, 
drawing off those students whose needs the other high schools were not able to meet. There 
had been no alternative but dismissal from school prior to the creation of CAP, and the 
support for this last-chance instructional setting was deep-seated. During its three years of 
SDDAP funding and now. in its fifth year, the Coleman alternative program has continued to 
emphasize its role as service provider to troubled and over-age students who are 
educationally at risk. Clearly, the program has enjoyed a special niche in the overall 
education-related service plan for students in West Texas. CAP has never competed with the 
other high schools, and the amount of resources it draws from each of the (now eight) 
participating school districts is small in comparison to the returns to those districts— namely, 
fewer student-related problems and staff frustrations. 
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The Denver-Lake project provides the counterpoint to both the Des Moines and 
Coleman efforts in respect to building bridges to existing programs. On the one hand, the 
approach was new to the middle school, as indeed were the 6th-grade students who were 
being enrolled at the site for the first time. On the other hand. Gold Team teachers were 
given opportunities and responsibilities as part of the program that the other teachers were 
not to enjoy. In addition, there seems not to have been any attempt made to rationalize the 
program activity in terms of benefits to the school as a whole. Simply stated, the approach 
taken in Denver-Lake with the Gold Team was almost pre-destined to earn the resentment of 
other school staff. This resentment grew more intense during the project period. What began 
as feelings of favoritism quickly accelerated into rumors that Gold Team staff would replace 
some elective teachers on a permanent basis. This in turn developed into suspicions of Gold 
Team teacher exclusivity; for example, the Gold Team's planning together during lunch 
periods was a sign they considered themselves to be above the rest of the teaching staff. 
(Ironically, these sessions became necessary in 1989-90 because, as it did in every year of 
the grant, the district reduced its funding for the Gold Team activity at the site.) Finally, 
pressure from the other teachers that Gold Team students were disciplined differently (and 
unfairly with respect to the other students) led to frequent calls for change in the project's 
policy of "internally" disciplining students. 

The experiences of these three projects suggest three strategies that may be important in 
bridging new education-related activities with current programs. First, the aims of a new 
program should be related to ongoing activities and communicated to all parties. In Des 
Moines, for example, SWS teachers were informed about just how the New Horizons 
component would provide incentives for their students and help to focus career exploration 
activities undertaken as a part of the SWS life-skills curriculum. In Coleman, the principals 
of the consortium's high schools were informed that CAP would be an alternative setting 
established for students who just were not succeeding in their regular programs. By contrast, 
the purpose of the Gold Team experiment in terms of the "Lake program" was never 
apparently described to the teaching staff at Denver's Lake Middle School. As a result, they 
were left to draw their own conclusions and saw only what seemed to be unfair privileges 
and exclusivity. 

A second strategy irwoUcs familiarizing other staff with the workings of the new 
program so that they may appreciate its challenges and be able to share in program-related 
discussions. At Denver-Lake, for example, an in-service orientation to the Gold Team 
program for all the teaching staff at the site would have helped eliminate confusion over 
teaching assignments for Gold Team students. Similarly, this sort of meeting might have 
demonstrated that the organizers of the Gold Team program were seriously interested in the 
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opinions and ideas of resident staff concerning how to niost effectively implement various 
program components. 

Finally, the third bridging strategy that might be suggested for new educational projects 
involves the active integration of new with ongoing school programs. Specifically, the 
danger lies in permitting new programs to become isolated. At the outset of the Denver-Lake 
project, for example, the principal might have established a regular schedule for briefings to 
staff on the progress of the Gold Team experiment. In addition, if the purpose of the 
experiment was to provide data on possible new approaches to the organization of the middle 
school experience at the site, the principal might have asked teachers from the regular 
program to observe from time to time in Gold Team classrooms and share their impressions. 
The principal, along with the Gold Team staff, might also have considered rotating one or 
two regular teachers into the Gold Team program as instructional staff for a week at a time 
to give them a chance to try out the instructional approach for themselves. In addition to 
strengthening links with the regular program, such a strategy also may have been of benefit" 
when the needs arose to replace Gold Team staff following the first two years of the 
program. 

Replicating Dropout Prevention Programs 

The SDDAP initiative encouraged potential demonstration sites to replicate programs 
that had been successful previously in other locations. Among the in-depth study sites for the 
evaluation, three sites attempted replications. In each case, the type of replication activity 
was distinctive. In Broward County, for example, three sites within the district sought to 
replicate a program that had been developed and demonstrated as effective at another of the 
district's schools. In Aiken, the program that was replicated in both alternative and regular 
school settings had previously been identified by the NDN as a proven model. In Hannibal, it 
was the accelerated learning (or accelerated schools) model that was tried in one elementary 
school. In every one of these cases, the individuals who were eventually to implement the 
replications at the sites did not seek out the opportunity and were not involved in planning for 
it. The sites in Broward County were selected by Area Superintendents and the proposal was 
developed by district staff. In Aiken, the model to be replicated was selected by district staff, 
as was the site of the alternative school and its principal. Hannibal's denwnstration proposal 
was developed by the district's superintendent, who then sought a school with the appropriate 
demographic characteristics to carry out the project. 

Each of these replication sites was funded at the levels requested, and at each of the sites 
upon notification of award training sessions were conducted in the models to be 
implemented. The three Broward County middle schools attended orientation sessions 
conducted by staff at the model site in the district, the Aiken alternative school and regular 
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school principals attended orientation sessions conducted by developers of the Diversified 
Education Experiences Program (DEEP), and the principal of Eugene Field Elementary 
School in Hannibal, together with several teachers from the site, attended a state- sponsored 
workshop on the accelerated schools program. Following these initial training sessions, each 
of tiie repUcation sites were provided with ongoing assistance. Staff of the Broward County 
model site visited the three replication sites in the district on an occasional basis during the 
first year of die grant period and participated in montiily meetings, which began sometime in 
tiie second year. In Aiken, a second workshop for all project staff was held with DEEP 
developers at the alternative school site during the fu-st year of tiie grant; numerous materials 
and teacher guides were also available that described how to implement tiiis instructional 
system. Finally, tiie Hannibal project, benefiting from state-level commitment to tiiis 
educational reform, was assigned an accelerated schools liaison person by tiie state 
coordinator who visited the site frequentiy. In addition, a consultant who was well versed in 
tiie accelerated schools philosophy and operating principles was living in Jefferson City and 
was available on an as-needed basis to the project. 

Our visits to these projects in 1992-93 found them continuing to implement tiie models 
selected back in 1988-89 (with only the Broward County project receiving new SDDAP 
funds beginning in 1991-92). Although tiie data gathered on student outcomes at tiiese sites 
provided rather weak evidence of program success, tiie staff at each site were faitiifuUy 
carrying out project activities as prescribed by the program models. In Broward County, for 
example, tiie teacher-counselor teams at each of the three schools were training peer tutors, 
meeting witii parents, and over-seeing tutoring and student follow-up sessions. In Aiken, the 
alternative school continued to operate and teachere were working with class groups on 
various pioject^a characteristic of DEEP-inspircd activity. Finally, in Hannibal, teacher 
cadres and tiie school steering committee (witii parent representation) remained active, 
altiiough tiiey had adapted to life without SDDAP funding by shortening the length of cadre 
meetings and holding them before the start of school on a weekly basis. What, tiien, have 
tiiese projects to say about encouraging successful replications of educational programs? 

Clearly, none of tiiese projects would likely be operating today were it not for (1) 
SDDAP funding froiti 1988-89 tiirough 1990-91 and (2) initial and (some) follow-up 
training in how to in-plement tiie model approaches. In addition, tiiree factors importantiy 
affected tiie replication activities at various of tiie sites and may have inhibited tiie 
effectiveness of tiie projects for students. These factors included: 

• tiic "fit" of tiie models to the replication sites 

• tiie extent of principal "buy-in" to tiie replications 

• "turf' considerations 
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Model Fit 

As noted earlier, in these three demonstration sites the models that were selected for 
replication were selected by persons other than those charged with implementation. In 
addition, the specific sites at which the implementations would occur were in all cases 
identified initially by persons other than site-based representatives (e.g., principals or 
teachers). Given this situation, it is natural to wonder just how well the various models fit in 
with the priorities, attitudes, and expectations of the school staff persons at the three sites. 

At a general level, it really did not seem to matter much that the replication task(s) were 
given to the study sites, since with the tasks came additional funds and the promise of 
different activities. At all the sites, for example, the staff who were charged with day-to-day 
replication of the programs evidenced commitment and a high-energy approach. During the 
funding period particularly, these staff we^e quite active in holding meetings, thinking of 
ways to "personalize" the models, and generally doing all that they could do to bring about 
positive changes for the youth involved. 

More specifically, however, one or two important questions may be raised about the 
lack of fit of these models to the replication sites. For example, at the Hannibal site, one of 
the areas that the teacher cadres wanted most to review and to modify was the curriculum, 
but the district's policies regarding matters of curriculum prevented any changes other than 
the timing of various learning objectives. Issues of buying and using different texts or 
offering different sorts of classes for students were simply not ones the cadres could seriously 
address, yet addressing such issues is exactly what accelerated schools cadres, or planning 
groups, are strongly encouraged to do. Similarly, in Aiken, DEEP was selected precisely 
because it provided a project-oriented focus to learning and classroom management, however 
the students who were sent to the alternative school by the various high schools in the area 
more often than not lacked basic communication and interpersonal skills and many exhibited 
violent tendencies. While projects or more activity-oriented lessons generally may well be 
effective in working with students at risk, it is at least debatable as to whether the developers 
of the DEEP model would have recommended implementation with such a uniformly 
challenging group of students. 

In Broward County, there were no indicators of lack of fit of the Model School 
Adjustment Program (MSAP) to the three replication sites. Policies at the schools were in no 
way at odds with program objectives, and sufficient tutors and students who were judged to 
need special services were willing to participate. Principals at various of the replication sites 
did note that their student populations probably represented tougher challenges to any new 
program than did the students at the original model site, but there was little evidence that 
these challenges were in any way too great for the program design. Cultural differences or 
household structures in the schools' service areas did not conflict or seem to militate against 
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the requirements regarding parental involvement in the program. In short, tne match between 
the model and the characteristics of the three Broward County replication sites seemed 
appropriate. 

Principal Buy-In 

Where the three Broward County replication sites did differ markedly from the original 
model site was in the extent of principal buy-in to the program. The three replication-site 
principals were quite willing to "house" the progrann., as they likely were (and would be) 
willing to house most any program that brought its own funding and would add to the 
available repertoire of services at the site, but they did evidence considerably diminished 
levels of personal involvement in the model program as compared to the original model site. 
Certainly the model- site principal had designed and pioneered the program (and continues to 
develop new components to enhance its operation), however the difference in overt and 
observable commitment may also be due to a difference in approach generally. Based on 
visits to the model site and the replication sites over the previous three years, the principal at 
the model site seemed much more hands-on in the teaching-leaming-parenting of the school; 
that is, the principal seemed often less a manager of school activities and more a front-line 
organizer and participant in them. If this sort of commitment to program on the part of the 
building principal is a necessary component of the MSAP, as it seems to us it may be, then 
the model should specify it and replication efforts must work actively to bring it about. 

In Aiken also, the commitment of principals to the DEEP replication was a factor in the 
reportedly rather poor performance of the regular-school version of the demonstration. 
Beginning with the orientation session, which the principals were required to attend and 
which they felt was a waste of time, their lack of enthusiasm could only exacerbate normal 
implementation challenges. When staff of the sites had to be selected for work on the regular- 
school DEEP implementation, for example, this lack of enthusiasm led to what was described 
as a grabbing of people to assign to DEEP responsibilities. In fact, the situation was made 
worse because SDDAP funding in the first and third years of the grant period was not 
received until relatively late in the academic year. To have organized regular-school DEEP 
activities in a shortened timeframe demanded special efforts that, without a sense of 
commnitment to the program, simply could not be expected. In addition, the lack of shared 
understanding among these individuals and the alternative program staff of what the 
alternative program was attempting likely contributed to the wholesale transfer of students to 
the program who were ill-prepared to succeed there. 

In the case of principal buy-in, Hannibal stands as perhaps the most positive example of 
what can be accomplished. When the superintendent approached the principal about 
implementing the accelerated schools model, the principal agreed, attended the training 
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sessions, and became enthused about what might be accomplished. This principal, who is 
only the fifth principal in the school's history dating back to the late 1800s, also found that 
the grant monies were a great incentive for teachers; it provided them with the release-time to 
become involved in school planning, and the principal not only was not threatened by this but 
genuinely seemed happy that people were getting involved. Taking a stand-back posture, the 
principal allowed the teachers to design and carry out several new (to the school) programs, 
ranging from extended day care to playground revitalization. Since the grant period ended, 
this principal, who rarely before had initiated any funding requests for starting new programs 
or continuing ones that had been developed at the site, has written several such requests. 
Turf 

"Turf' issues typically arise in the implementation of collaborative programs (e.g., 
interagency coordination efforts). They may also arise in cases where support or assistance is 
required across jurisdictional lines of authority. In the cases of both the Broward County and 
Aiken projects, they arose in the context of replicating model programs. Since the Hannibal 
site was the only replication site in the district, no such problems of turf manifested 
themselves. 

The issues of turf in Broward County were subtle, as were the manner in which they 
seemed to affect the replication of the MSAP. When the project was funded in 1988-89, staff 
at the original model site were pleased and excited about "their" program being tried in 
different schools. They were also careful about becoming personally involved in the 
replication activities for fear they would be seen as intruding or invading the turf of other 
school staff. As a result, their early formal involvement in the replication was no more 
pronounced than that of the developers of the DEEP program at the Aiken site; that is, they 
attended a meeting and talked about their program. To the extent that staff at the other three 
sites in Broward County sought out help from model-site staff it was on an individual basis 
and did not occur all that frequently. The modei-site principal, for example, was actually 
quite reluctant to contact the replication-site principals for the purpose of sharing ideas and 
tips related to the program. His concern was that, in this district's context of site-based 
management, such actions might well be taken as a sign of lack of confidence or, worse yet, 
as meddling in others' affairs. 

In Aiken, the problems associated with turf were less subtle. When a new principal from 
outside the district was hired for the alternative school and sought to become the district-wide 
DEEP coordinator, the other principals rejected the idea. V/hen the alternative school staff, 
through their principal, sought to modify the criteria for student selection for the alternative 
program, no changes were made in current practices. When the DEEP activities at the regular 
schools began to dissipate in the third year of the SDDAP funding period, there was little 
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attempt made at any of the buildings to sustain DEEP activities. In short, the principals at the 
regular high schools in the county had neither bought-in to the DEEP program or to the 
concept of an alternative site organized around DEEP principles. Most of all, they were not 
interested in or willing to altering their own schools' approaches to handling the problems of 
students at risk. 

A Sampler of Replication Strategies 

The experiences of the Broward County, Aiken, and Hannibal projects in replicating 
model programs suggest several strategies that may be useful for future replication efforts. 
First, there is the need to bring together district-level and school personnel at the outset of 
planning for replication efforts. The choice of the program to be replicated should, ideally, be 
discussed and agreed upon by all parties to be involved in the effort, and the choice of the 
replication site should probably be a product of these discussions. A second strategy is to 
encourage regular and ongoing interactions involving district-level and school personnel 
throughout the Ufe of the program— perhaps in the form of a general steering committee. 
Third, special efforts must be made to make the program especially meaningful to the 
principals at the replication sites. For example, these principals should be encouraged to 
attend training sessions related to the model program, and, whenever possible, they should be 
encouraged to think of ways they can use or expand upon the replication experience to gain 
support for their schools generally. 

A fourth strategy for enhancing a replication effort, when more than a single school is to 
be involved in that effort, is to decide upon a mechanism for coordinating activities across 
the sites. In Broward County, for example, a district coordinator for the MSA? projects now 
oversees the activities at the various sites and takes the lead in organizing cross-site 
meetings, training sessions, and more informal discussions. Related to the issue of 
coordination, a fifth strategy is to promote continual improvement in the replication effort. 
Specifically, persons with experience in the model to be replicated should be identified as 
special resources; ways should be found to encourage their visiting the replication sites on a 
regular basis over the life of the effort. In the case of the Broward County projects, for 
example, where turf considerations may have inhibited this type of sharing, district-level 
leadership in identifying model site staff as resources (i.e., persons with special credentials 
above and beyond their positions within the district personnel system) might very likely have 
led to greater information-sharing and brainstorming about how to adapt the MSAP to the 
new sites. 

Clearly there is overiap among these five strategies and their intended aims. Bringing 
staff together at the outset, for example, may indeed promote principal buy-in, and principal 
buy-in is a first step toward making it easier to establish a cross-site system of information- 
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sharing. The use of resource persons, particularly when they reside in the replication districts, 
seems such a natural step that we may often overtook the political and interperscr.al 
dynamics that often prevent such exchanges. These types of exchanges are especially critical, 
however, if we are to move beyond the copying of model programs to their further 
development. Indeed, one important goal of replicating any model dropout prevention 
program, for example, must be to enhance the robustness of that model through adaptation 
and extension of its features. 



• U.S. GOVEWiMENT PRINTING OFFICE : 1 ^ - 3 99- 9 5 6 /4 



